In this lesson, you will learn how to use the tracemalloc module to pinpoint exactly where memory is leaking in your long-running Python services.
When building high-concurrency systems or long-lived background processes, objects that are not properly garbage collected can accumulate, leading to "memory bloat" that eventually crashes your application. tracemalloc acts as a debugger for these invisible leaks by recording the exact source files and line numbers responsible for allocating memory.
How to Track Allocations
To detect a leak, you must start tracing, perform an operation, and then take a "snapshot" of memory usage. By comparing two snapshots, you can identify which objects were allocated but never released.
import tracemalloc
# Start tracking memory allocations
tracemalloc.start()
# Take an initial snapshot
snapshot1 = tracemalloc.take_snapshot()
# Perform your suspected leaking task
leaky_list = []
def run_task():
for _ in range(1000):
leaky_list.append("I am a leak!")
run_task()
# Take a second snapshot
snapshot2 = tracemalloc.take_snapshot()
# Compare snapshots to see what grew
top_stats = snapshot2.compare_to(snapshot1, 'lineno')
print("[ Top 5 differences ]")
for stat in top_stats[:5]:
print(stat)
Analyzing the Results
When you call compare_to, tracemalloc filters out objects that existed in both snapshots, leaving you with the "delta." If you see a specific line in your code consistently appearing at the top of the list with a high byte count, you have found the root of your leak.
In complex systems—especially those utilizing asyncio or process-based task queues—it is helpful to trigger these snapshots periodically within your monitoring logic. By logging these differences over time, you can distinguish between expected memory usage (like a growing cache) and actual leaks caused by hanging references or circular dependencies that the garbage collector cannot clear.
Key Takeaway: Use tracemalloc snapshots to compare memory states before and after specific tasks, allowing you to isolate and resolve the exact line of code preventing objects from being garbage collected.