In this lesson, you will learn how to move inactive records to "cold storage" while keeping them discoverable through a search-optimized index.
The Problem with "Hot" Data
As your application grows, your primary database collection (the "hot" tier) gets cluttered with years of old user activity. Keeping everything in one place slows down queries and increases storage costs. However, simply deleting old data isn't an option because you often need to audit past actions or let users search their entire history.
The Archival Strategy: Move and Reference
To solve this, we implement a "cold storage" pattern. Instead of deleting, we move inactive documents to a separate collection (e.g., archived_orders) or a cheaper storage medium. To maintain searchability, we leave behind a "stub"—a tiny document in the main collection that contains only the search fields and a reference ID to the cold storage.
Implementation Pattern
Imagine you have an orders collection. When an order is older than one year, you archive it:
// 1. Copy the full record to the archive collection
await db.collection('archived_orders').insertOne(fullOrder);
// 2. Replace the main document with a "stub"
await db.collection('orders').replaceOne(
{ _id: orderId },
{
_id: orderId,
search_keywords: fullOrder.keywords, // Keep fields needed for search
archived: true,
archived_at: new Date()
}
);
When a user searches for an order, your application queries the main collection. If the search result has the archived: true flag, the application knows to fetch the full details from archived_orders instead of the active collection.
Why This Works
This strategy keeps your "hot" collection lean, ensuring that index scans remain lightning-fast. Because you kept a stub in the main collection, your existing full-text search indexes still point to these records, maintaining the "searchability" requirement without the performance penalty of housing huge, inactive documents in your primary memory or index space.
Key Takeaway: Archive by moving the heavy payload to a secondary collection while leaving a search-optimized "stub" in your primary collection to maintain visibility.