Course contentsShow
← AI Engineering
Lesson 972 of 1,88624. Building AI APIsFree lesson

Multiple Model Endpoints

Design API patterns for serving multiple models: separate endpoints vs. model selection via request parameters.

Multiple Model Endpoints

What you'll learn: How to architect APIs that serve multiple AI models, comparing separate endpoints versus model selection through request parameters.

The Core Challenge

When your application needs to serve multiple LLM models (e.g., GPT-4 for complex tasks, GPT-3.5 for simple ones, or different specialized models), you face an architectural decision: should each model get its own endpoint, or should one endpoint handle routing based on request parameters?

Pattern 1: Separate Endpoints

Create distinct routes for each model:

@app.post("/api/v1/chat/gpt4")
async def chat_gpt4(request: ChatRequest):
    return await llm_client.generate(model="gpt-4", ...)

@app.post("/api/v1/chat/gpt3")
async def chat_gpt3(request: ChatRequest):
    return await llm_client.generate(model="gpt-3.5-turbo", ...)

Pros: Clear separation, easy versioning, dedicated monitoring per model
Cons: More routes to maintain, harder to add new models dynamically

Pattern 2: Model Selection via Parameters

Use one endpoint with a model selector:

class ChatRequest(BaseModel):
    message: str
    model: str = "gpt-3.5-turbo"

@app.post("/api/v1/chat")
async def chat(request: ChatRequest):
    return await llm_client.generate(model=request.model, ...)

Pros: Single endpoint, flexible model selection, easier client code
Cons: Requires validation logic, mixed monitoring metrics

When to Use Which

Use separate endpoints when models have different SLAs, pricing, or response schemas. Use parameter-based selection when models are interchangeable and share interfaces—think of it like choosing shirt sizes from one store versus visiting different stores.

Key Takeaway: Separate endpoints provide isolation and clarity; parameter-based selection offers flexibility and simplicity. Choose based on whether your models are fundamentally different services or variations of the same capability.