Course contentsShow
← AI Engineering
Lesson 1137 of 1,88628. Prompt and Chain OptimizationFree lesson

Profiling Tools for Python and API Calls

Use cProfile, line_profiler, and HTTP timing headers to measure execution time across code paths.

Profiling Tools for Python and API Calls

What you'll learn: How to precisely measure where time is spent in your LLM application code and API interactions.

Why Profiling Matters

When optimizing LLM chains, gut feelings aren't enough. You need hard data showing whether your bottleneck is Python code, network calls, or model processing. Think of profiling like a stopwatch at each checkpoint in a race—you discover exactly where the runner slows down.

Core Profiling Tools

cProfile: Function-Level Timing

Python's built-in cProfile shows how much time each function consumes. Run your script with:

import cProfile
import pstats

profiler = cProfile.Profile()
profiler.enable()

# Your LLM chain code here
result = generate_with_retrieval(query)

profiler.disable()
stats = pstats.Stats(profiler)
stats.sort_stats('cumulative')
stats.print_stats(10)  # Top 10 slowest functions

This reveals cumulative time spent in each function—perfect for finding which components dominate your chain's execution.

line_profiler: Line-by-Line Analysis

For deeper investigation, line_profiler shows time per line within specific functions:

from line_profiler import LineProfiler

lp = LineProfiler()
lp.add_function(my_embedding_function)
lp.run('my_embedding_function(documents)')
lp.print_stats()

This pinpoints exact bottleneck lines, like slow loops or redundant transformations.

HTTP Timing Headers

API providers often return timing headers. Inspect response headers for metrics like X-Request-ID or custom timing data:

response = requests.post(api_url, json=payload)
print(f"Server time: {response.headers.get('X-Response-Time')}")
print(f"Round-trip: {response.elapsed.total_seconds()}s")

This separates network latency from server processing time.

Practical Application

Profile first, optimize second. Measure both Python overhead and external API calls separately to determine if you need faster code, better caching, or parallel requests.

Key Takeaway: Use cProfile for function-level analysis, line_profiler for line-by-line detail, and HTTP headers to separate network from processing time—measure everything before optimizing anything.