Course contentsShow
← AI Engineering
Lesson 996 of 1,88624. Building AI APIsFree lesson

Chunked Transfer Encoding

Use HTTP chunked encoding to send progressive responses without buffering.

Chunked Transfer Encoding

What you'll learn: How to send LLM streaming responses efficiently without waiting for the complete answer using HTTP chunked transfer encoding.

The Core Idea

When streaming LLM responses, you don't know the total response size upfront—tokens arrive progressively from the model. Chunked Transfer Encoding is an HTTP mechanism that lets your server send data in pieces (chunks) without declaring a Content-Length header beforehand. Each chunk is sent as soon as it's ready, allowing the client to receive and display tokens in real-time.

Think of it like a pizza delivery service that brings slices as they come out of the oven, rather than waiting to bake the entire pizza before delivering anything.

How It Works

In a chunked response:

  1. Your server sets Transfer-Encoding: chunked in the response headers
  2. Each chunk is prefixed with its size in hexadecimal
  3. Chunks are sent as LLM tokens arrive
  4. A final zero-length chunk signals completion

Unlike Server-Sent Events (SSE) which adds a text-based protocol on top, chunked encoding is a lower-level HTTP transport mechanism. SSE typically uses chunked encoding under the hood.

Practical Example

from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

async def token_generator():
    tokens = ["Hello", " world", "!", " From", " LLM"]
    for token in tokens:
        yield token  # Each yield becomes a chunk

@app.get("/stream")
async def stream_response():
    return StreamingResponse(
        token_generator(),
        media_type="text/plain"
    )
    # FastAPI automatically uses chunked encoding for StreamingResponse

The client receives each token immediately without buffering the entire response server-side.

Key Takeaway: Chunked transfer encoding is the HTTP transport layer that enables streaming responses when the total size is unknown, making it essential for real-time LLM token delivery without memory overhead.