Course contentsShow
← AI Engineering
Lesson 93 of 1,8863. API-First AI EngineeringFree lesson

Stop Sequences and Max Tokens Configuration

Learn to control generation length and termination using max_tokens and custom stop sequences.

Stop Sequences and Max Tokens Configuration

What you'll learn: How to precisely control when and where an LLM stops generating text using max_tokens and custom stop sequences.

Why Control Generation Length?

When calling LLM APIs, you need boundaries. Without them, the model might generate thousands of tokens, costing you money and time. Two mechanisms give you this control: max_tokens (a hard limit) and stop sequences (smart termination signals).

Max Tokens: The Hard Ceiling

Think of max_tokens as a word count limit on an essay. If you set max_tokens=100, the model will generate at most 100 tokens before stopping—even mid-sentence. This prevents runaway generation and caps your per-request cost.

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Explain photosynthesis"}],
    max_tokens=50  # Stop after 50 tokens maximum
)

Stop Sequences: Smart Termination

Stop sequences are custom strings that tell the model "when you generate this, stop immediately." This is like telling a storyteller, "Stop when you say 'The End.'"

Common use cases:

  • Structured output: Stop at "\n\n" to get one paragraph
  • List generation: Stop at "###" to separate sections
  • Dialogue systems: Stop at "User:" to prevent the model from role-playing both sides
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "List three benefits:"}],
    stop=["\n\n", "###"]  # Stop at double newline OR ###
)

The model checks after each generated token. When it encounters any stop sequence, generation halts, and that sequence isn't included in the response.

Key Takeaway: Use max_tokens to cap costs and prevent unlimited generation; use stop sequences to create predictable, structured outputs by teaching the model where to naturally end.