Profiling Tools for Python and API Calls
What you'll learn: How to precisely measure where time is spent in your LLM application code and API interactions.
Why Profiling Matters
When optimizing LLM chains, gut feelings aren't enough. You need hard data showing whether your bottleneck is Python code, network calls, or model processing. Think of profiling like a stopwatch at each checkpoint in a race—you discover exactly where the runner slows down.
Core Profiling Tools
cProfile: Function-Level Timing
Python's built-in cProfile shows how much time each function consumes. Run your script with:
import cProfile
import pstats
profiler = cProfile.Profile()
profiler.enable()
# Your LLM chain code here
result = generate_with_retrieval(query)
profiler.disable()
stats = pstats.Stats(profiler)
stats.sort_stats('cumulative')
stats.print_stats(10) # Top 10 slowest functions
This reveals cumulative time spent in each function—perfect for finding which components dominate your chain's execution.
line_profiler: Line-by-Line Analysis
For deeper investigation, line_profiler shows time per line within specific functions:
from line_profiler import LineProfiler
lp = LineProfiler()
lp.add_function(my_embedding_function)
lp.run('my_embedding_function(documents)')
lp.print_stats()
This pinpoints exact bottleneck lines, like slow loops or redundant transformations.
HTTP Timing Headers
API providers often return timing headers. Inspect response headers for metrics like X-Request-ID or custom timing data:
response = requests.post(api_url, json=payload)
print(f"Server time: {response.headers.get('X-Response-Time')}")
print(f"Round-trip: {response.elapsed.total_seconds()}s")
This separates network latency from server processing time.
Practical Application
Profile first, optimize second. Measure both Python overhead and external API calls separately to determine if you need faster code, better caching, or parallel requests.
Key Takeaway: Use cProfile for function-level analysis, line_profiler for line-by-line detail, and HTTP headers to separate network from processing time—measure everything before optimizing anything.