Course contentsShow
← AI Engineering
Lesson 962 of 1,88623. LLM Application Architecture PatternsFree lesson

Security and Privacy in Caching

Handling sensitive data in caches, user isolation, and encryption for multi-tenant systems.

Security and Privacy in Caching

You'll learn how to protect sensitive user data when caching LLM requests and responses in multi-tenant applications.

The Challenge

Caching saves money and improves speed, but it creates a serious risk: cached responses might leak between users. Imagine caching "What's my account balance?" — if User A's cached response appears for User B, you've just exposed private financial data.

Core Principles

User Isolation

Never use cache keys that could match across different users. Always include a user identifier or tenant ID in your cache key:

# Bad: cache_key = hash(prompt)
# Good: cache_key = hash(user_id + prompt)

Think of it like hotel room keys — each guest needs their own unique key, even if they ask the front desk the same question.

Encryption at Rest

Sensitive cached data should be encrypted in your cache store (like Redis). This protects against:

  • Database breaches
  • Unauthorized access to the cache server
  • Snapshot or backup exposure

Encryption in Transit

Use TLS/SSL for all connections to your cache. Never transmit unencrypted prompts or responses containing PII (personally identifiable information) over the network.

Multi-Tenant Considerations

In systems serving multiple organizations, implement namespace isolation. Each tenant gets a separate cache namespace, preventing cross-tenant contamination:

cache_key = f"{tenant_id}:{user_id}:{hash(prompt)}"

Selective Caching

Not everything should be cached. Skip caching for:

  • Requests containing passwords, tokens, or payment details
  • Regulated data (HIPAA, GDPR-sensitive content)
  • User-specific queries that reference private information

Key Takeaway: Security in caching requires user isolation through proper key design, encryption both at rest and in transit, and careful decisions about what data is safe to cache in multi-tenant LLM applications.