Course contentsShow
← AI Engineering
Lesson 1311 of 1,88632. Fine-Tuning for ProductionFree lesson

Model Capability Gaps vs Training Needs

Distinguish between tasks requiring better base models versus tasks where fine-tuning can bridge capability gaps.

Model Capability Gaps vs Training Needs

What you'll learn: How to determine whether a task failure stems from fundamental model limitations or simply needs better training data through fine-tuning.

The Core Distinction

Not every AI system shortcoming can be fixed with fine-tuning. Some tasks require capabilities that a base model simply doesn't possess, while others are performance issues that training can address.

Model capability gaps are fundamental limitations in what a model can do—like asking a small language model to perform complex multi-step reasoning, or expecting a text-only model to understand images. These require choosing a different, more capable base model.

Training needs are situations where the model could perform the task but needs examples to learn your specific requirements—like adopting your company's writing style, following domain-specific formatting rules, or using specialized terminology correctly.

The Decision Framework

Think of it like hiring: capability gaps are "this person doesn't have the required skills," while training needs are "this person has the skills but needs onboarding."

Signs of a capability gap:

  • The model fails at the core task even with perfect prompts
  • No amount of examples improve base performance
  • The task requires reasoning depth beyond the model's architecture
  • Success requires modalities (vision, audio) the model doesn't support

Signs of a training need:

  • The model gets the general idea but misses specifics
  • Failures follow consistent patterns you've identified
  • You have quality examples of correct behavior
  • The task is similar to what the model already does well

This distinction matters for both cost and feasibility—upgrading models is expensive but sometimes necessary, while fine-tuning is your tool for customization within existing capabilities.

Key Takeaway: Before investing in fine-tuning, honestly assess whether your model architecturally can do the task—fine-tuning teaches style and specifics, but can't add fundamental reasoning or multimodal capabilities.