When to Use Grammar-Based vs JSON Mode
You'll learn how to choose between grammar-based generation and JSON mode based on your specific structured output needs.
The Core Decision
Both grammar-based generation (like Guidance, Outlines, or GBNF) and JSON mode (with or without function calling) help you get structured output from LLMs, but they excel in different scenarios.
JSON Mode is your default choice when:
- You need common data formats (JSON objects, arrays, nested structures)
- Your schema is relatively simple and well-defined
- You're using cloud APIs (OpenAI, Anthropic) that natively support JSON mode
- Performance and speed matter most (JSON mode is typically faster)
- You want easy integration with Pydantic models for validation
- Your output format is flexible enough to allow minor variations
Grammar-Based Generation shines when:
- You need very strict format guarantees (no tolerance for malformed output)
- Your output follows non-JSON patterns (CSV, custom syntax, code with specific structure)
- You're running local models (llama.cpp) where you control token generation
- You need complex constraints beyond what JSON schemas express
- 100% conformance is critical (no post-generation validation needed)
- You're dealing with domain-specific languages or structured formats
The Analogy
Think of JSON mode like ordering from a menu—you pick standard dishes and sometimes customize them. Grammar-based generation is like having a chef who physically cannot serve anything except what's explicitly allowed in your recipe book.
Key Takeaway: Use JSON mode for speed and simplicity with standard data structures; use grammar-based generation when you need absolute format guarantees or non-JSON structured output, especially with local models.