Course contentsShow
← AI Engineering
Lesson 140 of 1,8864. Prompt Engineering FoundationsFree lesson

Max Tokens and Length Control

Setting generation limits to control response length and manage costs effectively.

Max Tokens and Length Control

What you'll learn: How to set generation limits that control how long your AI's responses are—and keep your costs predictable.

Why Length Matters

When you call an LLM API, the model could theoretically keep generating text indefinitely (or until it decides to stop). That's both a billing risk and a user experience problem. The max_tokens parameter lets you cap how many tokens the model can generate in a single response.

Think of it like giving someone a word limit for an essay. If you say "write up to 500 words," they'll stop there even if they have more to say.

How Max Tokens Works

The max_tokens parameter specifies the maximum number of tokens the model can generate. This doesn't include your input prompt—only the output.

For example:

  • max_tokens=50 → short, concise answer
  • max_tokens=500 → detailed explanation
  • max_tokens=2000 → comprehensive response

If the model reaches the limit mid-sentence, it simply stops. You'll often see the finish_reason field in the API response indicate "length" rather than "stop" (natural completion).

Balancing Cost and Quality

Since you pay per token, max_tokens directly controls cost. Set it too low, and answers get cut off. Set it too high, and you waste money on padding or overly verbose responses.

Best practice: Start with a reasonable estimate based on your use case (summaries = 150–300 tokens; full articles = 1000+), then adjust based on actual output.

Key Takeaway: max_tokens is your primary tool for controlling response length, managing costs, and preventing runaway generation—set it thoughtfully based on your task requirements.