Course contentsShow
← Python
Lesson 878 of 1,06014. Concurrency and PerformanceFree lesson

Managing Shared Memory Using Process Synchronization Primitives

Implement locks and semaphores to prevent race conditions when multiple processes attempt to access or modify shared resources safely.

In this lesson, you will learn how to orchestrate safe access to shared memory when working with multiple processes using synchronization primitives like locks and semaphores.

When you use the multiprocessing module to bypass the Global Interpreter Lock (GIL) and achieve true parallel computation, you often need to share data between processes using objects like multiprocessing.Value or multiprocessing.Array. However, because these processes operate in separate memory spaces, simply reading and writing to these variables is not atomic. If two processes attempt to increment a shared counter at the same time, you encounter a race condition, where updates can be lost or overwritten.

To resolve this, we use process-aware synchronization primitives. Unlike threading primitives, these objects are designed to work across process boundaries by communicating with the operating system kernel.

Implementing a Lock for Mutual Exclusion

A Lock ensures that only one process can enter a "critical section" of your code at any given time. If a second process attempts to acquire the lock while it is already held, it will block until the first process releases it.

from multiprocessing import Process, Value, Lock

def increment(shared_val, lock):
    with lock:  # Only one process can enter here at a time
        temp = shared_val.value
        shared_val.value = temp + 1

# Shared memory counter
counter = Value('i', 0)
lock = Lock()

processes = [Process(target=increment, args=(counter, lock)) for _ in range(10)]
for p in processes: p.start()
for p in processes: p.join()

print(f"Final counter value: {counter.value}")

Managing Resource Pools with Semaphores

While a Lock allows exactly one process access, a Semaphore manages a fixed-size pool of resources by maintaining an internal counter. When you initialize a Semaphore(n), up to n processes can acquire the resource simultaneously. This is ideal for limiting concurrent access to a database or a file that can handle a specific number of simultaneous connections but would crash under heavy load. By wrapping your memory operations with acquire() and release() calls, you ensure your shared resources remain protected without bottlenecking your entire parallel pipeline.

Key Takeaway: Use process-safe locks and semaphores to enforce atomicity when multiple processes modify shared memory, ensuring that shared state remains consistent despite concurrent execution.