Setting Up Your First LLM Endpoint
What you'll learn: How to create a functional POST endpoint that accepts user prompts and returns LLM-generated completions using proper request/response models.
The Core Idea
An LLM endpoint is the gateway between your users and your AI model. Think of it as a restaurant order window: customers submit requests (prompts), and the kitchen (LLM) delivers completed dishes (responses). Your endpoint handles the intake, validation, and delivery of this exchange.
Using FastAPI (which you've already learned), you'll create a POST route that:
- Accepts incoming prompts with structured data models
- Calls your LLM (OpenAI, Anthropic, etc.)
- Returns the completion in a clean, typed response
The Structure
from fastapi import FastAPI
from pydantic import BaseModel
from openai import OpenAI
app = FastAPI()
client = OpenAI()
class PromptRequest(BaseModel):
prompt: str
max_tokens: int = 100
class CompletionResponse(BaseModel):
completion: str
model: str
@app.post("/generate", response_model=CompletionResponse)
async def generate_completion(request: PromptRequest):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": request.prompt}],
max_tokens=request.max_tokens
)
return CompletionResponse(
completion=response.choices[0].message.content,
model=response.model
)
This endpoint validates inputs automatically (thanks to Pydantic), ensures type safety, and provides clear API documentation through FastAPI's auto-generated docs.
Why This Matters
Proper request/response models prevent malformed data, make your API self-documenting, and catch errors before they reach your LLM (saving costs and time). It's the foundation for reliable AI services.
Key Takeaway: A well-structured LLM endpoint uses typed request/response models to create a reliable, documented, and maintainable interface between users and your AI system.