Security and Privacy in Caching
You'll learn how to protect sensitive user data when caching LLM requests and responses in multi-tenant applications.
The Challenge
Caching saves money and improves speed, but it creates a serious risk: cached responses might leak between users. Imagine caching "What's my account balance?" — if User A's cached response appears for User B, you've just exposed private financial data.
Core Principles
User Isolation
Never use cache keys that could match across different users. Always include a user identifier or tenant ID in your cache key:
# Bad: cache_key = hash(prompt)
# Good: cache_key = hash(user_id + prompt)
Think of it like hotel room keys — each guest needs their own unique key, even if they ask the front desk the same question.
Encryption at Rest
Sensitive cached data should be encrypted in your cache store (like Redis). This protects against:
- Database breaches
- Unauthorized access to the cache server
- Snapshot or backup exposure
Encryption in Transit
Use TLS/SSL for all connections to your cache. Never transmit unencrypted prompts or responses containing PII (personally identifiable information) over the network.
Multi-Tenant Considerations
In systems serving multiple organizations, implement namespace isolation. Each tenant gets a separate cache namespace, preventing cross-tenant contamination:
cache_key = f"{tenant_id}:{user_id}:{hash(prompt)}"
Selective Caching
Not everything should be cached. Skip caching for:
- Requests containing passwords, tokens, or payment details
- Regulated data (HIPAA, GDPR-sensitive content)
- User-specific queries that reference private information
Key Takeaway: Security in caching requires user isolation through proper key design, encryption both at rest and in transit, and careful decisions about what data is safe to cache in multi-tenant LLM applications.