Course contentsShow
← AI Engineering
Lesson 328 of 1,8869. Retrieval-Augmented Generation (RAG) - BasicsFree lesson

RAG vs Prompt Stuffing

Understanding when to use RAG's indexed retrieval versus directly including context in prompts for small datasets.

RAG vs Prompt Stuffing

What you'll learn: When to use indexed retrieval versus simply pasting context directly into your prompts.


The Core Question

You have 10 customer support documents. Should you build a RAG pipeline with embeddings and vector search, or just copy all 10 into your prompt? This is prompt stuffing vs RAG.

Prompt stuffing means directly including all your context in the prompt itself:

Here are all our policies:
[Policy 1: 500 words]
[Policy 2: 450 words]
...
Answer this question: [user query]

RAG means indexing those documents, retrieving only the relevant ones, then including just those in your prompt.

When Prompt Stuffing Makes Sense

  • Small datasets (< 5-10 documents, under ~2000 tokens total)
  • Every document is equally relevant to every query
  • Simplicity is paramount — no infrastructure needed
  • One-time or infrequent tasks

It's like carrying all your business cards in your pocket instead of organizing a filing system. Fine if you only have 10 cards.

When RAG Wins

  • Large knowledge bases (hundreds+ documents)
  • Only some docs answer each query — why pay LLM costs to process irrelevant context?
  • Exceeds context window limits
  • Frequently changing content — update the index, not every prompt
  • Performance matters — smaller prompts = faster, cheaper responses

Think of it as the difference between reading an entire encyclopedia to answer one question versus using the index to find the right page.

Key Takeaway: Use prompt stuffing for tiny, static datasets where everything is always relevant. Use RAG when your knowledge base is large, dynamic, or when you need to retrieve selectively to stay within token budgets and reduce costs.