Measuring ML Energy Consumption
You'll learn how to quantify the energy and carbon costs of your machine learning workflows.
Why Measurement Matters
Understanding energy consumption is the first step toward sustainable AI. Just as you track model accuracy and training time, energy metrics help you understand the environmental footprint of your work—from training massive models to serving billions of predictions.
Core Measurement Approaches
Hardware-Level Monitoring
Modern GPUs and TPUs expose power draw through APIs (like NVIDIA's SMI tools). These report instantaneous wattage, which you integrate over time to calculate total energy in kilowatt-hours (kWh). Think of it like a car's fuel gauge—you need continuous readings to know total consumption.
Software-Based Tools
Libraries like CodeCarbon, experiment-impact-tracker, and cloud provider dashboards automate energy tracking. They monitor your training scripts, recording power draw across all hardware components (GPU, CPU, RAM, cooling), then convert energy to carbon emissions using regional electricity grid data.
Scope of Measurement
Don't just measure training—track:
- Training: The obvious energy sink, especially for large models
- Inference: Often overlooked, but serving millions of predictions adds up
- Data storage: Maintaining datasets in data centers requires constant power
- Hyperparameter search: Multiple training runs multiply your footprint
Key Metrics
- Energy (kWh): Total electricity consumed
- Carbon emissions (kg CO₂eq): Energy × grid carbon intensity
- PUE (Power Usage Effectiveness): Data center efficiency factor (cooling, lighting overhead)
The Measurement Challenge
Energy varies wildly by hardware, location (clean vs. coal grids), and task. A single training run might consume 100–1000+ kWh. Without measurement, you're flying blind on optimization opportunities.
Key Takeaway: Measuring energy consumption across training, inference, and storage using hardware monitors and software tools is essential for understanding and reducing ML's environmental impact.