Course contentsShow
← Machine Learning and Deep Learning
Lesson 1917 of 3,53841. Structured Output and Tool CallingFree lesson

Handling Malformed JSON Outputs

Learn strategies to parse, repair, and retry when LLMs produce invalid or incomplete JSON responses.

Handling Malformed JSON Outputs

What you'll learn: You'll discover practical strategies to detect, repair, and recover from invalid or incomplete JSON responses generated by LLMs.

Even with JSON mode and Pydantic validation (concepts you've learned), LLMs occasionally produce malformed JSON—missing brackets, trailing commas, unquoted keys, or incomplete structures. Rather than letting your application crash, you need robust error-handling strategies.

Why JSON Fails

LLMs generate tokens sequentially and may:

  • Stop mid-generation due to token limits
  • Hallucinate invalid syntax under ambiguous prompts
  • Mix natural language commentary with JSON
  • Produce inconsistent quote styles or escape sequences

Think of it like a student turning in homework that's almost right—you don't fail them immediately; you guide them to fix it.

Core Strategies

1. Parse and Catch

Always wrap JSON parsing in try-except blocks. Log the raw output for debugging.

2. Partial Completion Detection

Check if the response ends abruptly (no closing brace). You can attempt to "close" the structure programmatically by adding missing brackets.

3. Regex-Based Extraction

Strip surrounding text before/after the actual JSON block. Look for patterns like {...} or [...] and extract just that portion.

4. Retry with Corrections

Send the malformed output back to the LLM with an instruction: "This JSON is invalid. Fix it: [malformed output]". The model often corrects its own mistakes.

5. Fallback Prompts

If retries fail, switch to a simpler schema or ask for plain text, then manually structure it.

6. Gradual Degradation

Accept partial results—if 8 of 10 fields are valid, use those and mark the rest as missing rather than discarding everything.

Key Takeaway:
Malformed JSON is inevitable with LLMs; robust applications combine defensive parsing, automated repair attempts, and graceful degradation to maintain reliability.