Course contentsShow
← AI Engineering
Lesson 964 of 1,88624. Building AI APIsFree lesson

Setting Up Your First LLM Endpoint

Create a basic POST endpoint that accepts prompts and returns LLM completions with proper request/response models.

Setting Up Your First LLM Endpoint

What you'll learn: How to create a functional POST endpoint that accepts user prompts and returns LLM-generated completions using proper request/response models.

The Core Idea

An LLM endpoint is the gateway between your users and your AI model. Think of it as a restaurant order window: customers submit requests (prompts), and the kitchen (LLM) delivers completed dishes (responses). Your endpoint handles the intake, validation, and delivery of this exchange.

Using FastAPI (which you've already learned), you'll create a POST route that:

  1. Accepts incoming prompts with structured data models
  2. Calls your LLM (OpenAI, Anthropic, etc.)
  3. Returns the completion in a clean, typed response

The Structure

from fastapi import FastAPI
from pydantic import BaseModel
from openai import OpenAI

app = FastAPI()
client = OpenAI()

class PromptRequest(BaseModel):
    prompt: str
    max_tokens: int = 100

class CompletionResponse(BaseModel):
    completion: str
    model: str

@app.post("/generate", response_model=CompletionResponse)
async def generate_completion(request: PromptRequest):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": request.prompt}],
        max_tokens=request.max_tokens
    )
    
    return CompletionResponse(
        completion=response.choices[0].message.content,
        model=response.model
    )

This endpoint validates inputs automatically (thanks to Pydantic), ensures type safety, and provides clear API documentation through FastAPI's auto-generated docs.

Why This Matters

Proper request/response models prevent malformed data, make your API self-documenting, and catch errors before they reach your LLM (saving costs and time). It's the foundation for reliable AI services.

Key Takeaway: A well-structured LLM endpoint uses typed request/response models to create a reliable, documented, and maintainable interface between users and your AI system.