Prompt Template Design for RAG
What you'll learn: How to craft prompt templates that seamlessly combine system instructions, retrieved documents, and user questions to get accurate LLM responses.
The Challenge: Three Pieces, One Cohesive Request
Now that you've retrieved relevant documents and formatted them, you need to assemble a prompt that guides the LLM effectively. A RAG prompt template is like a recipe card—it tells the model how to use the ingredients (retrieved context) to answer the question.
A good RAG prompt template has three essential components:
1. System Instructions
The "rules of engagement" for the model. This might include:
- "Answer based only on the provided context"
- "If the context doesn't contain the answer, say so"
- "Be concise" or "Provide detailed explanations"
2. Retrieved Context
The formatted documents you retrieved (from the previous lesson). Clearly delineated with markers like ### Context: or Documents: so the model knows where the facts are.
3. User Query
The original question, placed strategically—often at the end—with a clear prompt like ### Question: or Based on the above context, answer:
Template Structure Example
You are a helpful assistant. Answer using only the context below.
### Context:
{retrieved_documents}
### Question:
{user_query}
### Answer:
Key Takeaway: A well-designed RAG prompt template explicitly separates instructions, context, and questions with clear markers, ensuring the LLM knows its role, the information source, and what you're asking—maximizing accuracy while respecting context window limits.