Course contentsShow
← AI Engineering
Lesson 995 of 1,88624. Building AI APIsFree lesson

Server-Sent Events (SSE) for Streaming

Implement SSE to stream LLM responses token-by-token to clients in real-time.

Server-Sent Events (SSE) for Streaming

What you'll learn: How to stream LLM responses token-by-token to clients in real-time using Server-Sent Events.

Why SSE for LLM Streaming?

When an LLM generates a response, waiting for the entire completion before showing anything creates a poor user experience. Imagine watching a blank screen for 10 seconds versus seeing words appear as they're generated—like a person typing. SSE enables this real-time streaming effect.

Server-Sent Events is a standard protocol where the server pushes data to the client over a single HTTP connection. Unlike WebSockets (which are bidirectional), SSE is one-way: perfect for streaming LLM tokens from server to client.

How SSE Works for LLM Streaming

Think of SSE as a TV broadcast: the server (station) sends a continuous stream of data (program), and clients (viewers) tune in to receive it. The connection stays open until the stream completes.

When streaming LLM responses:

  1. Client makes a request with Accept: text/event-stream
  2. Server immediately responds with Content-Type: text/event-stream
  3. As the LLM generates each token, you send it as an SSE event
  4. Client receives and displays tokens progressively
  5. Server closes connection when generation completes

Each SSE message follows this format:

data: {"token": "Hello", "done": false}\n\n
data: {"token": " world", "done": false}\n\n
data: {"token": "!", "done": true}\n\n

The double newline (\n\n) signals the end of each event.

Key Characteristics

  • Built on standard HTTP (no special protocols)
  • Automatic reconnection if connection drops
  • Works through most firewalls and proxies
  • Simpler than WebSockets for one-way streaming
  • Compatible with FastAPI's StreamingResponse

Key Takeaway: SSE provides a simple, standard way to stream LLM tokens to clients in real-time, dramatically improving perceived responsiveness without complex bidirectional protocols.