Handling Soft Deletes in Document Stores

In this lesson, you will learn how to implement "soft deletes" to safely remove data from your collection while maintaining historical integrity and ensuring your active queries remain performant.

The Problem with Permanent Deletion

In many business applications, simply removing a document via a hard delete operation is dangerous. You might need to recover accidentally deleted data, audit user activity, or maintain referential integrity for reporting. However, if you keep everything, your active queries—like fetching "all active orders"—become slower because they must filter through mountains of "dead" records.

Implementing Soft Deletes

The soft delete pattern involves adding a flag, such as isDeleted: true or a deletedAt: timestamp, to your document. Instead of removing the document, you perform an atomic field update to toggle this status.

When you want to retrieve only the live data, you modify your queries to exclude those marked as deleted. To keep performance high, you should combine this flag with a partial index.

Code Illustration

Suppose you have a collection of products. To "delete" an item without losing it, you update the document:

// Perform a soft delete via atomic update
db.products.updateOne(
  { _id: productId },
  { $set: { isDeleted: true, deletedAt: new Date() } }
);

Now, to maintain query performance, you create a partial index that only includes documents where isDeleted is not true:

// Create a partial index for high-performance active queries
db.products.createIndex(
  { name: 1 }, 
  { partialFilterExpression: { isDeleted: { $ne: true } } }
);

Optimizing for Performance

By using a partial index, the database engine ignores the "deleted" documents entirely during index lookups. This keeps your index size small and your search speed fast, even as your collection accumulates a history of stale documents. When you need to perform an audit or "undelete" a record, you simply query the full collection or unset the flag.

This approach balances the need for data retention with the requirement for lightning-fast application performance.

Key Takeaway: Use a boolean flag combined with a partial index to hide deleted records from your application while preserving the data for audit and recovery.