Course contentsShow
← Machine Learning and Deep Learning
Lesson 1214 of 3,53827. Pretrained Language Models: GPT Family and BeyondFree lesson

Evolution of Training Techniques Across GPT Generations

Tracing improvements in optimization, data curation, and compute efficiency from GPT-1 to GPT-4.

Evolution of Training Techniques Across GPT Generations

What you'll learn: How OpenAI improved optimization, data quality, and compute efficiency from GPT-1 through GPT-4.

The Journey of Refinement

Each GPT generation didn't just scale up parameters—the way they trained evolved dramatically. Think of it like moving from a single-oven bakery (GPT-1) to an industrial facility with specialized equipment (GPT-4): not just bigger, but smarter.

GPT-1: Foundation Setting

GPT-1 established the decoder-only architecture and next-token prediction objective. Training was relatively straightforward: standard Adam optimizer, basic learning rate schedules, and limited compute (~1 month on 8 GPUs). Data came from BooksCorpus—high quality but limited scale.

GPT-2: Scaling and Diversity

GPT-2 introduced WebText—a curated 40GB dataset scraped from Reddit links, prioritizing quality over raw size. Training techniques remained similar, but careful data curation became the differentiator. They proved that diverse, filtered web text beat simply "more data."

GPT-3: Compute Efficiency at Scale

With 175B parameters, GPT-3 required massive optimization innovations: mixed-precision training (FP16), gradient checkpointing to save memory, and distributed training across thousands of GPUs. The dataset grew to hundreds of billions of tokens with weighted sampling from different sources (books, web, Wikipedia) to balance quality.

GPT-3.5 and Beyond: Human Alignment

InstructGPT introduced Reinforcement Learning from Human Feedback (RLHF), fundamentally changing the training paradigm. Models were first pretrained, then fine-tuned on instructions, then refined using human preferences—a three-stage pipeline.

GPT-4: Multimodal and Efficient

GPT-4 brought suspected Mixture of Experts architectures for compute efficiency (activating only relevant model portions per token) and extended training to handle both text and images through unified pretraining objectives.

Key Takeaway: From GPT-1 to GPT-4, the evolution wasn't just about "bigger"—it was about smarter data curation, memory-efficient techniques, distributed optimization, and human-aligned objectives that transformed raw language modeling into aligned, capable assistants.