Course contentsShow
← AI Engineering
Lesson 538 of 1,88613. LLM Orchestration FrameworksFree lesson

Debugging Framework-Wrapped Calls

Techniques for tracing through framework abstractions to understand what's actually being sent to models.

Debugging Framework-Wrapped Calls

What you'll learn: How to peek under the hood when frameworks like LangChain or LlamaIndex make API calls on your behalf.

The Problem with Black Boxes

When you use a framework, it wraps your LLM calls in multiple layers of abstraction. You write chain.invoke(input) and get a response, but what exactly was sent to the model? Which prompt template was used? What parameters? If something goes wrong, you're debugging through a black box.

Think of it like ordering food through a delivery app versus calling the restaurant directly. The app is convenient, but if your order is wrong, you need to trace through the app's interface, the restaurant's system, and the driver to figure out where things went wrong.

Core Tracing Techniques

Enable verbose logging: Most frameworks have a verbose=True flag that prints intermediate steps:

chain = LLMChain(llm=llm, prompt=prompt, verbose=True)

Use callbacks: Frameworks like LangChain expose callback handlers that intercept every API call:

from langchain.callbacks import StdOutCallbackHandler
chain.invoke(input, config={"callbacks": [StdOutCallbackHandler()]})

Inspect the trace: Look for the framework's tracing utilities (like LangChain's langchain.debug = True) to see the full execution path.

Check network logs: Use tools like httpx debugging or browser dev tools to see the actual HTTP requests leaving your application—the raw JSON payload tells the truth.

Print intermediate objects: Before invoking, print the prompt template after variable substitution to verify what text will be sent.

When to Dig Deeper

Trace framework calls when responses seem wrong, costs spike unexpectedly, or performance degrades. The framework might be concatenating context differently than you expect or making redundant calls.

Key Takeaway: Debugging frameworks means turning on verbose logging, using callbacks, and occasionally inspecting raw network traffic to see exactly what's being sent to the LLM beneath the abstraction layer.