Prompt Caching Fundamentals
You'll learn how to leverage prompt caching in LLM APIs to dramatically reduce costs when repeatedly sending the same context.
What Is Prompt Caching?
Modern LLM providers (like Anthropic's Claude and OpenAI's GPT models) offer prompt caching: the ability to store and reuse portions of your prompt that don't change between requests. Instead of reprocessing the same system instructions, examples, or context every time, the API caches the processed "prefix" and only charges you for:
- Cache writes (first time processing)
- Cache reads (reusing cached content - typically 90% cheaper)
- New tokens (the varying part of your prompt)
How It Works
Think of prompt caching like a restaurant prep station. If you're making 100 sandwiches with the same bread and lettuce, you prep those once and keep them ready. Only the unique toppings get added fresh each time.
When you send a prompt with cacheable content:
- The provider processes and stores the unchanging prefix (system prompt, long context, examples)
- For subsequent requests with the same prefix, the API retrieves the cached processing state
- Only the new/changing portion gets processed from scratch
When to Use It
Prompt caching shines when you have:
- Fixed system prompts used across many requests
- Large context documents that remain constant (documentation, codebase excerpts)
- Static few-shot examples
- Repeated conversational context
Cache validity typically lasts 5-10 minutes of inactivity, making it ideal for user sessions and batch processing jobs.
Key Takeaway: Prompt caching reduces costs by 50-90% when you structure prompts so unchanging content (system instructions, context) comes first and gets cached, while only variable content gets processed fresh each request.