Data integrity checksumming

In this lesson, you will learn how to leverage modern filesystems to detect and automatically recover from "silent data corruption," also known as bit rot.

The Problem: Silent Corruption

Standard filesystems like Ext4 ensure metadata integrity through journaling, but they don't necessarily verify that the actual user data (your photos, documents, and code) remains unchanged on the physical disk. Over time, sectors can degrade, or hardware errors can flip bits without the OS realizing there is a problem. You might open a file, read corrupted data, and save it back, effectively destroying your backup.

The Solution: Checksumming

Modern "copy-on-write" filesystems, specifically Btrfs, solve this by storing a checksum for every block of data.

1. Checksum Calculation: When you write data to the disk, the filesystem calculates a mathematical "fingerprint" (a CRC32C checksum) of that data and stores it in the filesystem's metadata tree.
2. Verification on Read: Every time you read that file back, the filesystem re-calculates the checksum of the block it just retrieved and compares it to the original stored fingerprint.
3. Detection and Recovery: If the checksums do not match, the filesystem knows the data is corrupted. If you have redundant storage—such as a Btrfs RAID1 or DUP profile—the system will automatically fetch the valid copy from the other disk or location and fix the corrupted block in the background.

Seeing it in Action

You don't need to enable this; if you are using Btrfs, it is active by default. You can inspect the health of your data by running:

# Scrub the filesystem to verify checksums across the entire volume
sudo btrfs scrub start /mnt/data
# Check the status of a running scrub
sudo btrfs scrub status /mnt/data

The scrub process walks through all data blocks and verifies them against the stored checksums. If it encounters a mismatch, it reports the "csum errors" and attempts an automatic repair if redundancy is available. By moving from a passive storage model to an active, self-verifying one, you ensure that your files remain as they were when they were first written.

Key Takeaway: Data integrity checksumming proactively identifies bit rot by comparing real-time data fingerprints against stored metadata, enabling filesystems like Btrfs to self-heal corrupted blocks.