Course contentsShow
← AI Engineering
Lesson 45 of 1,8862. Working with Pre-trained ModelsFree lesson

Model Variants and Checkpoints

Understanding base models vs fine-tuned variants, and selecting checkpoints for specific domains or languages.

Model Variants and Checkpoints

What you'll learn: How to distinguish between base models and their fine-tuned variants, and choose the right checkpoint for your specific use case.

The Model Family Tree

When you find a model on the Hugging Face Hub (like BERT or GPT-2), you're usually looking at a family of related models, not just one. Think of it like smartphone models: there's the base iPhone, then variants optimized for different needs—iPhone Pro for cameras, iPhone Mini for portability.

Base Models vs Fine-Tuned Variants

A base model is trained on general data without targeting any specific task. It's like a student who's completed general education but hasn't specialized yet.

A fine-tuned variant takes that base model and continues training on specific data for particular tasks, domains, or languages. Examples:

  • bert-base-uncased → base model
  • bert-base-uncased-finetuned-sst2 → specialized for sentiment analysis
  • camembert-base → BERT fine-tuned specifically for French

Understanding Checkpoints

A checkpoint is a saved snapshot of a model at a specific point in its training. You'll often see multiple checkpoints for the same model representing different training stages or configurations. Choose based on:

  • Domain: medical text? Use BioBERT. Legal documents? Try Legal-BERT
  • Language: multilingual base models work everywhere but language-specific variants usually perform better
  • Task alignment: if someone already fine-tuned for your exact task, start there

The more specialized the variant, the better it performs on similar data—but the worse on different tasks.

Key Takeaway: Base models offer flexibility; fine-tuned variants offer performance. Always check if a domain-specific or task-specific checkpoint exists before using the general base model.