Course contentsShow
← AI Engineering
Lesson 110 of 1,8863. API-First AI EngineeringFree lesson

Handling Partial Responses and Deltas

Process incremental token updates, accumulate partial responses, and maintain state during streaming.

Handling Partial Responses and Deltas

What you'll learn: How to process incomplete chunks of text as they arrive during streaming and build complete responses from fragments.

When you enable streaming with an LLM API (using Server-Sent Events), the model doesn't send you one complete response—it sends many tiny pieces called deltas. Each delta contains just a few tokens, sometimes even single characters. Your job is to catch these fragments, store them, and reassemble them into the full answer.

The Challenge of Incremental Updates

Think of streaming responses like receiving a letter one word at a time through a mail slot. Each envelope (delta) contains just a word or two. You need to:

  1. Open each envelope as it arrives (parse the delta)
  2. Read what's inside (extract the new token/text)
  3. Append it to your growing letter (accumulate state)
  4. Keep track of where you are (maintain position and context)

Maintaining State During Streaming

Unlike batch responses where you get everything at once, streaming requires you to maintain state—a variable that holds all accumulated text so far. Each time a new delta arrives, you add its content to your running total.

The delta typically contains only the new content, not everything repeated. For example:

  • Delta 1: "The"
  • Delta 2: " weather"
  • Delta 3: " is"
  • Accumulated result: "The weather is"

You must handle the case where deltas arrive out of order, connection drops mid-stream, or the final delta signals completion.

Key Takeaway: Streaming responses require you to accumulate partial deltas into complete text by maintaining state across multiple incremental updates, not just waiting for one finished response.