Course contentsShow
← Python
Lesson 396 of 1,0606. Error Handling and File I/OFree lesson

Atomic File Writes for Data Safety

Ensure data consistency by implementing atomic write patterns that prevent partial file corruption during unexpected system failures or power interruptions.

In this lesson, you will learn how to prevent file corruption by using the "write-replace" pattern to ensure your data is always either fully saved or not saved at all.

The Problem: Partial Writes

When you open a file in Python and start writing to it, the operating system doesn't necessarily commit all that data to the disk at once. If your program crashes, the computer loses power, or the disk runs out of space while you are mid-write, you end up with a "partial file." This file is often corrupted, leaving your application in a broken state because the previous version of the data is destroyed, and the new version is incomplete.

The Solution: Atomic Operations

To achieve "atomicity," we avoid overwriting our target file directly. Instead, we use an intermediate temporary file to prepare our data.

The strategy follows these three steps:

  1. Write the new data to a temporary file in the same directory as your target file.
  2. Ensure the data is flushed and synced to the physical disk (using os.fsync()).
  3. Use the os.replace() function to rename the temporary file to the final destination name.

Because os.replace() is an atomic system call, the operating system guarantees that the file name will point to either the old version or the new version, but never a corrupted hybrid of the two.

Example Implementation

import os
import tempfile

def atomic_write(filename, data):
    # 1. Create a temp file in the same directory
    dir_name = os.path.dirname(filename)
    with tempfile.NamedTemporaryFile('w', dir=dir_name, delete=False) as tf:
        tf.write(data)
        tf.flush()
        # 2. Force the OS to write to disk
        os.fsync(tf.fileno())
        temp_name = tf.name
    
    # 3. Atomically replace the target file with the temp file
    os.replace(temp_name, filename)

By using this pattern, you treat the file system as an "all-or-nothing" storage mechanism. If the process dies before os.replace() is called, your original file remains untouched. If it dies after, you have the complete new file.

Key Takeaway: Never overwrite an existing file directly; instead, write to a temporary file and use os.replace() to swap it into place to guarantee data consistency.