Background Tasks for Logging
What you'll learn: How to use FastAPI background tasks to capture logs and metrics after sending your API response, keeping your endpoints lightning-fast.
The Core Problem
When you build an LLM API, you need to log everything—user requests, model responses, token counts, latency, errors. But if you write logs synchronously in your endpoint handler, every database insert or file write adds milliseconds (or worse) to your user's wait time. For AI endpoints that already take seconds, this compounds the problem.
How Background Tasks Work
FastAPI's BackgroundTasks lets you schedule functions to run after the HTTP response is sent to the client. The user gets their answer immediately, while logging happens behind the scenes.
Think of it like a restaurant: the waiter brings your food right away (response sent), then cleans up and updates inventory later (background task). You don't wait for the cleanup.
Practical Implementation
Add BackgroundTasks as a dependency in your endpoint, then use .add_task():
from fastapi import BackgroundTasks
def log_usage(user_id: str, prompt: str, tokens: int):
# Write to database or file
db.insert_log(user_id, prompt, tokens)
@app.post("/generate")
async def generate(
request: PromptRequest,
background_tasks: BackgroundTasks
):
response = await llm.generate(request.prompt)
# Schedule logging without waiting
background_tasks.add_task(
log_usage,
request.user_id,
request.prompt,
response.token_count
)
return response
The client receives response instantly. Logging happens afterward, even if it takes 500ms.
Key Takeaway: Background tasks decouple slow logging from fast responses, letting you capture comprehensive metrics without punishing user experience.