Multiple Model Endpoints
What you'll learn: How to architect APIs that serve multiple AI models, comparing separate endpoints versus model selection through request parameters.
The Core Challenge
When your application needs to serve multiple LLM models (e.g., GPT-4 for complex tasks, GPT-3.5 for simple ones, or different specialized models), you face an architectural decision: should each model get its own endpoint, or should one endpoint handle routing based on request parameters?
Pattern 1: Separate Endpoints
Create distinct routes for each model:
@app.post("/api/v1/chat/gpt4")
async def chat_gpt4(request: ChatRequest):
return await llm_client.generate(model="gpt-4", ...)
@app.post("/api/v1/chat/gpt3")
async def chat_gpt3(request: ChatRequest):
return await llm_client.generate(model="gpt-3.5-turbo", ...)
Pros: Clear separation, easy versioning, dedicated monitoring per model
Cons: More routes to maintain, harder to add new models dynamically
Pattern 2: Model Selection via Parameters
Use one endpoint with a model selector:
class ChatRequest(BaseModel):
message: str
model: str = "gpt-3.5-turbo"
@app.post("/api/v1/chat")
async def chat(request: ChatRequest):
return await llm_client.generate(model=request.model, ...)
Pros: Single endpoint, flexible model selection, easier client code
Cons: Requires validation logic, mixed monitoring metrics
When to Use Which
Use separate endpoints when models have different SLAs, pricing, or response schemas. Use parameter-based selection when models are interchangeable and share interfaces—think of it like choosing shirt sizes from one store versus visiting different stores.
Key Takeaway: Separate endpoints provide isolation and clarity; parameter-based selection offers flexibility and simplicity. Choose based on whether your models are fundamentally different services or variations of the same capability.