Max Tokens and Length Control
What you'll learn: How to set generation limits that control how long your AI's responses are—and keep your costs predictable.
Why Length Matters
When you call an LLM API, the model could theoretically keep generating text indefinitely (or until it decides to stop). That's both a billing risk and a user experience problem. The max_tokens parameter lets you cap how many tokens the model can generate in a single response.
Think of it like giving someone a word limit for an essay. If you say "write up to 500 words," they'll stop there even if they have more to say.
How Max Tokens Works
The max_tokens parameter specifies the maximum number of tokens the model can generate. This doesn't include your input prompt—only the output.
For example:
max_tokens=50→ short, concise answermax_tokens=500→ detailed explanationmax_tokens=2000→ comprehensive response
If the model reaches the limit mid-sentence, it simply stops. You'll often see the finish_reason field in the API response indicate "length" rather than "stop" (natural completion).
Balancing Cost and Quality
Since you pay per token, max_tokens directly controls cost. Set it too low, and answers get cut off. Set it too high, and you waste money on padding or overly verbose responses.
Best practice: Start with a reasonable estimate based on your use case (summaries = 150–300 tokens; full articles = 1000+), then adjust based on actual output.
Key Takeaway: max_tokens is your primary tool for controlling response length, managing costs, and preventing runaway generation—set it thoughtfully based on your task requirements.