Course contentsShow
← AI Engineering
Lesson 1770 of 1,88643. Workflow Orchestration and State MachinesFree lesson

Workflow Timeouts and Circuit Breakers

Implement timeouts for individual steps and overall workflows to prevent indefinite execution.

Workflow Timeouts and Circuit Breakers

What you'll learn: How to prevent multi-step AI workflows from hanging indefinitely by implementing timeouts and circuit breakers that fail fast when things go wrong.

Why Workflows Need Guardrails

In multi-step AI workflows, each step—whether it's an LLM call, embedding generation, or image processing—can potentially hang or take far longer than expected. Without timeouts, a single stuck API call can freeze your entire workflow indefinitely, wasting resources and frustrating users.

Think of it like a restaurant kitchen: if one station takes too long preparing a dish, you don't let the entire service grind to a halt. You set time limits and have backup plans.

Implementing Timeouts

Step-level timeouts set maximum execution time for individual operations. If an LLM call is expected to take 10 seconds, set a 15-second timeout. When exceeded, the step fails immediately rather than waiting forever.

Workflow-level timeouts govern the entire execution. Even if individual steps succeed within their limits, the overall workflow might have a 2-minute ceiling. This prevents cascading delays from accumulating.

Circuit Breakers: Failing Fast

A circuit breaker monitors repeated failures and temporarily stops attempting operations that are consistently failing. After three consecutive timeout failures from a particular API, the circuit "opens"—immediately rejecting new requests without even trying. This prevents wasted resources hammering a dead service.

After a cooldown period, the circuit enters a "half-open" state, testing if the service has recovered. One success closes the circuit; another failure reopens it.

This pattern protects both your system and downstream dependencies from overload during outages.

Key Takeaway: Timeouts and circuit breakers are essential safety mechanisms that prevent individual failures from cascading through your entire workflow, enabling graceful degradation and faster recovery in production AI systems.