Course contentsShow
← AI Engineering
Lesson 583 of 1,88614. Function Calling and Tool UseFree lesson

Timeout and Resource Limits

Implement execution timeouts, rate limits, and resource constraints to prevent runaway function calls.

Timeout and Resource Limits

What you'll learn: How to protect your application from runaway function calls by setting execution timeouts, rate limits, and resource constraints.

Why You Need Limits

When an LLM calls functions or tools, things can go wrong: a function might hang indefinitely, make too many API calls, or consume excessive memory. Without guardrails, a single bad function call could freeze your application or rack up unexpected costs. Just like a circuit breaker prevents electrical overload, timeout and resource limits prevent your AI system from spiraling out of control.

Core Implementation Strategies

Execution Timeouts

Set a maximum duration for each function call. If a function exceeds this limit, terminate it gracefully. This prevents infinite loops or slow external API calls from blocking your application.

Rate Limits

Control how many function calls can occur within a time window (e.g., max 10 calls per minute). This protects against LLMs that repeatedly call expensive tools or hammer external APIs.

Resource Constraints

Monitor and cap memory usage, file system access, or network bandwidth during function execution. This is especially important when allowing LLMs to execute potentially resource-intensive operations.

Practical Application

When implementing timeout logic (already covered in "Multi-turn Timeout and Limits"), you're now extending that concept to individual function calls. Before executing any tool, wrap it with these constraints. If limits are exceeded, return a controlled error message to the LLM rather than crashing.

Think of it like a restaurant with timed seating: each party gets a reasonable time slot, and the kitchen limits how many orders it accepts simultaneously. This keeps service flowing smoothly for everyone.

Key Takeaway: Implementing timeouts, rate limits, and resource constraints prevents runaway function calls from degrading system performance, exhausting resources, or incurring unexpected costs.