Generation Methods: generate()
You'll learn how to use the generate() method to make pre-trained language models produce coherent text automatically.
What is generate()?
The generate() method is the workhorse for autoregressive text generation in Hugging Face Transformers. "Autoregressive" means the model predicts text one token at a time, using what it's already written to inform what comes next—like typing a sentence where each word depends on the previous ones.
While .forward() (from Model Outputs and Hidden States) gives you raw predictions for a single step, generate() handles the entire loop: predicting a token, adding it to your prompt, predicting the next token, and repeating until you have complete text.
How It Works
Think of generate() like autocomplete on steroids. You provide a starting phrase (tokenized input), and the model keeps suggesting the next most likely word, then treats that suggestion as part of the input for the next prediction. It continues this cycle until hitting a stopping condition (max length or a special "end" token).
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
inputs = tokenizer("The future of AI is", return_tensors="pt")
output_tokens = model.generate(**inputs, max_length=20)
generated_text = tokenizer.decode(output_tokens[0])
print(generated_text)
The method returns token IDs, which you decode back into readable text using your tokenizer.
Key Takeaway: generate() orchestrates the full autoregressive loop for producing coherent text from pre-trained language models, making text generation a single method call instead of manual iteration.