Course contentsShow
← AI Engineering
Lesson 114 of 1,8863. API-First AI EngineeringFree lesson

Cancellation and Timeout Management

Handle user-initiated cancellations, implement request timeouts, and clean up streaming connections.

Cancellation and Timeout Management

What you'll learn: How to stop streaming requests gracefully when users change their mind or when requests take too long.

Why This Matters

When streaming AI responses (the pattern you just learned), users may click "stop" mid-generation, or your system needs to enforce maximum wait times. Without proper cancellation and timeout handling, you'll waste tokens, tie up connections, and create a poor user experience.

Core Concepts

User-Initiated Cancellation

Think of streaming like a phone call — you need a way to "hang up" cleanly. When a user clicks stop in your UI, you must:

  • Signal the streaming connection to close
  • Stop processing incoming chunks
  • Release any held resources (memory, open connections)

With most API clients, this means aborting the HTTP request before the stream completes naturally.

Request Timeouts

Not all requests should run forever. Setting timeouts prevents:

  • Runaway costs from unexpectedly long generations
  • Hanging connections that block other requests
  • Poor user experience from infinite waiting

You set a maximum duration, and if the stream hasn't completed by then, you terminate it.

Connection Cleanup

Whether cancelled or timed out, proper cleanup means:

  • Closing the SSE connection gracefully
  • Logging partial responses for debugging
  • Updating UI state to show "stopped" vs "completed"
  • Freeing memory from buffered chunks

Without cleanup, your application leaks connections and memory over time.

Key Takeaway: Streaming responses need explicit cancellation and timeout logic to handle user interruptions, enforce time limits, and prevent resource leaks — treating streaming as an active connection you must manage, not just fire-and-forget.