Course contentsShow
← AI Engineering
Lesson 1199 of 1,88629. Cost OptimizationFree lesson

Multi-Tier Model Architectures

Design systems with small, medium, and large models to balance cost and capability across workloads.

Multi-Tier Model Architectures

What you'll learn: How to design systems that route requests across small, medium, and large models to optimize both cost and performance.

The Core Idea

After classifying whether queries are simple or complex (lesson 1198), you need an architecture that acts on that classification. A multi-tier model architecture organizes your models into tiers based on their capability and cost—typically small (fast, cheap), medium (balanced), and large (powerful, expensive)—and routes each request to the most appropriate tier.

Think of it like a hospital triage system: not every patient needs a specialist. Routine questions go to a nurse (small model), moderate cases to a general practitioner (medium model), and only complex cases to a specialist (large model). This maximizes efficiency while maintaining quality care.

Designing Your Tiers

Tier 1 (Small): Handle 60-80% of simple queries with models like GPT-3.5-turbo, Claude Haiku, or fine-tuned small models. Use for FAQ, classification, simple extraction.

Tier 2 (Medium): Handle moderately complex reasoning with models like GPT-4-mini or mid-sized options. Use for multi-step reasoning, nuanced analysis.

Tier 3 (Large): Reserve for complex reasoning, creative tasks, or when accuracy is critical. Models like GPT-4, Claude Opus handle these expensive operations.

Architecture Pattern

Your routing layer (from lesson 1197) directs traffic based on query classification. Each tier can have its own caching strategy, rate limits, and fallback rules. Monitor tier distribution—if 90% hits Tier 3, your classifier needs tuning. If Tier 1 has high error rates, queries might be misrouted.

Key Takeaway: Multi-tier architectures distribute workloads across models of varying capability, dramatically reducing costs by reserving expensive models for truly complex tasks while maintaining quality through intelligent routing.