Input Length and Token Limit Validation
What you'll learn: How to validate prompt length and estimate token counts to prevent context window overflow before calling LLMs.
Why This Matters
LLMs have fixed context windows (e.g., 4K, 8K, 128K tokens). If you send a prompt that exceeds this limit, the API call will fail, wasting time, money, and user patience. Validating input length upfront prevents these failures and provides immediate, clear feedback to users.
The Core Problem
Text length in characters ≠ token count. A 1,000-character string might be 250 tokens or 400 tokens depending on the vocabulary. Different models use different tokenizers (GPT-4 vs Claude vs Llama), so "how long is too long?" varies.
Analogy: Imagine packing a suitcase with a weight limit. You can't just guess by looking—you need to weigh it. Similarly, you can't just count characters; you must estimate or calculate tokens.
Practical Validation Steps
- Character-based quick check: Set a conservative character limit (e.g., 3,000 chars for a 1K token limit) as a fast pre-filter
- Token estimation: Use the model's tokenizer library (like
tiktokenfor OpenAI models) to count tokens accurately - Reserve buffer: Leave room for system prompts, response tokens, and safety margin (e.g., use 90% of max)
- Return clear errors: When validation fails, tell users exactly how many tokens they exceeded
Integration with Pydantic
Combine this with Pydantic validators (from lesson 975-976) to enforce limits before the request reaches your LLM client. Raise a ValidationError with helpful messages like "Input is 5,200 tokens; maximum is 4,000."
Key Takeaway: Always validate token counts before calling LLMs using model-specific tokenizers, not just character counts—this prevents expensive API failures and provides better user experience.