Debugging Framework-Wrapped Calls
What you'll learn: How to peek under the hood when frameworks like LangChain or LlamaIndex make API calls on your behalf.
The Problem with Black Boxes
When you use a framework, it wraps your LLM calls in multiple layers of abstraction. You write chain.invoke(input) and get a response, but what exactly was sent to the model? Which prompt template was used? What parameters? If something goes wrong, you're debugging through a black box.
Think of it like ordering food through a delivery app versus calling the restaurant directly. The app is convenient, but if your order is wrong, you need to trace through the app's interface, the restaurant's system, and the driver to figure out where things went wrong.
Core Tracing Techniques
Enable verbose logging: Most frameworks have a verbose=True flag that prints intermediate steps:
chain = LLMChain(llm=llm, prompt=prompt, verbose=True)
Use callbacks: Frameworks like LangChain expose callback handlers that intercept every API call:
from langchain.callbacks import StdOutCallbackHandler
chain.invoke(input, config={"callbacks": [StdOutCallbackHandler()]})
Inspect the trace: Look for the framework's tracing utilities (like LangChain's langchain.debug = True) to see the full execution path.
Check network logs: Use tools like httpx debugging or browser dev tools to see the actual HTTP requests leaving your application—the raw JSON payload tells the truth.
Print intermediate objects: Before invoking, print the prompt template after variable substitution to verify what text will be sent.
When to Dig Deeper
Trace framework calls when responses seem wrong, costs spike unexpectedly, or performance degrades. The framework might be concatenating context differently than you expect or making redundant calls.
Key Takeaway: Debugging frameworks means turning on verbose logging, using callbacks, and occasionally inspecting raw network traffic to see exactly what's being sent to the LLM beneath the abstraction layer.