Handling Malformed JSON Outputs
What you'll learn: You'll discover practical strategies to detect, repair, and recover from invalid or incomplete JSON responses generated by LLMs.
Even with JSON mode and Pydantic validation (concepts you've learned), LLMs occasionally produce malformed JSON—missing brackets, trailing commas, unquoted keys, or incomplete structures. Rather than letting your application crash, you need robust error-handling strategies.
Why JSON Fails
LLMs generate tokens sequentially and may:
- Stop mid-generation due to token limits
- Hallucinate invalid syntax under ambiguous prompts
- Mix natural language commentary with JSON
- Produce inconsistent quote styles or escape sequences
Think of it like a student turning in homework that's almost right—you don't fail them immediately; you guide them to fix it.
Core Strategies
1. Parse and Catch
Always wrap JSON parsing in try-except blocks. Log the raw output for debugging.
2. Partial Completion Detection
Check if the response ends abruptly (no closing brace). You can attempt to "close" the structure programmatically by adding missing brackets.
3. Regex-Based Extraction
Strip surrounding text before/after the actual JSON block. Look for patterns like {...} or [...] and extract just that portion.
4. Retry with Corrections
Send the malformed output back to the LLM with an instruction: "This JSON is invalid. Fix it: [malformed output]". The model often corrects its own mistakes.
5. Fallback Prompts
If retries fail, switch to a simpler schema or ask for plain text, then manually structure it.
6. Gradual Degradation
Accept partial results—if 8 of 10 fields are valid, use those and mark the rest as missing rather than discarding everything.
Key Takeaway:
Malformed JSON is inevitable with LLMs; robust applications combine defensive parsing, automated repair attempts, and graceful degradation to maintain reliability.