Course contentsShow
← AI Engineering
Lesson 447 of 1,88611. RAG - Improving GenerationFree lesson

Debugging Prompt Construction

Inspect the final prompt sent to the LLM, including system instructions, retrieved context, and user query formatting.

Debugging Prompt Construction

What you'll learn: How to inspect the exact prompt your RAG system sends to the LLM to identify formatting issues, missing context, or instruction problems.

The Hidden Message Problem

When your RAG system produces poor outputs, the problem often lies in what you think you're asking versus what the LLM actually receives. The final prompt sent to the LLM combines system instructions, retrieved context, user queries, and formatting—and any issues in this assembly can derail your results.

Think of it like debugging a SQL query: you wouldn't trust it blindly. You'd print the actual query string to see if your template substitution worked correctly. The same applies here.

What to Inspect

1. System Instructions: Are they present? Correctly formatted? Do they still instruct the model to use retrieved context and cite sources?

2. Retrieved Context: Is it actually injected? In the right position (before or after the query)? Properly delimited with markers or XML tags?

3. User Query: Is it preserved exactly as intended? Does it include conversation history if needed?

4. Formatting Issues: Extra whitespace, missing newlines, broken delimiters, or truncated content due to token limits?

How to Debug

Log or print the complete prompt before sending it to the LLM. Compare it against your template expectations. Check token counts—does the prompt fit within your context window? Verify that citations instructions appear where the model will see them, and that retrieved documents are clearly separated.

If outputs lack citations, inspect whether citation instructions made it into the final prompt. If answers ignore context, check whether context appears before the query as intended.

Key Takeaway: Always inspect the fully-constructed prompt sent to your LLM—template bugs, missing context, and formatting errors are invisible until you examine the actual string or message payload.