Course contentsShow
← Machine Learning and Deep Learning
Lesson 1314 of 3,53829. Core NLP TasksFree lesson

Controlling Generation Length and Stopping

Understand stopping criteria using special tokens, maximum length constraints, and early stopping conditions for generation tasks.

Controlling Generation Length and Stopping

You'll learn how models know when to stop generating text and how to control the length of their outputs.

The Problem

When a language model generates text, it could theoretically continue forever. Imagine asking someone to finish a story—without rules, they might never stop talking! We need mechanisms to tell the model "that's enough."

Three Primary Stopping Mechanisms

1. Special End-of-Sequence Tokens

Models are trained to emit a special EOS (End-of-Sequence) token when they believe generation is complete. Think of it like a period at the end of a sentence, but for the entire output.

# Conceptual: model generates until it outputs <EOS>
generated_tokens = ["The", "sky", "is", "blue", "<EOS>"]
# Generation stops here

When the model produces this token, decoding halts naturally.

2. Maximum Length Constraints

You set a hard limit: "generate no more than N tokens." This acts as a safety net preventing runaway generation even if the model never produces an EOS token.

max_length = 50  # Stop after 50 tokens maximum

This is crucial for controlling computational cost and response time.

3. Early Stopping Conditions

In beam search (which maintains multiple candidate sequences), you can stop when:

  • A certain number of complete sequences (those ending in EOS) have been found
  • The best possible incomplete sequence can't beat existing complete ones

Practical Considerations

Too short: Model might cut off mid-thought if max_length is restrictive.
Too long: Without limits, models waste compute or generate repetitive, low-quality text.
No EOS: Some models or poorly fine-tuned ones might not reliably produce EOS tokens, making max_length essential.

Key Takeaway: Generation stopping combines special tokens (EOS) for natural endings, maximum length constraints for safety, and early stopping strategies for efficiency—giving you control over when and how model output concludes.