Course contentsShow
← Machine Learning and Deep Learning
Lesson 3468 of 3,53876. Ethics and Societal ImpactFree lesson

Measuring ML Energy Consumption

Learn methods and tools for tracking energy usage during training, inference, and data storage.

Measuring ML Energy Consumption

You'll learn how to quantify the energy and carbon costs of your machine learning workflows.

Why Measurement Matters

Understanding energy consumption is the first step toward sustainable AI. Just as you track model accuracy and training time, energy metrics help you understand the environmental footprint of your work—from training massive models to serving billions of predictions.

Core Measurement Approaches

Hardware-Level Monitoring

Modern GPUs and TPUs expose power draw through APIs (like NVIDIA's SMI tools). These report instantaneous wattage, which you integrate over time to calculate total energy in kilowatt-hours (kWh). Think of it like a car's fuel gauge—you need continuous readings to know total consumption.

Software-Based Tools

Libraries like CodeCarbon, experiment-impact-tracker, and cloud provider dashboards automate energy tracking. They monitor your training scripts, recording power draw across all hardware components (GPU, CPU, RAM, cooling), then convert energy to carbon emissions using regional electricity grid data.

Scope of Measurement

Don't just measure training—track:

  • Training: The obvious energy sink, especially for large models
  • Inference: Often overlooked, but serving millions of predictions adds up
  • Data storage: Maintaining datasets in data centers requires constant power
  • Hyperparameter search: Multiple training runs multiply your footprint

Key Metrics

  • Energy (kWh): Total electricity consumed
  • Carbon emissions (kg CO₂eq): Energy × grid carbon intensity
  • PUE (Power Usage Effectiveness): Data center efficiency factor (cooling, lighting overhead)

The Measurement Challenge

Energy varies wildly by hardware, location (clean vs. coal grids), and task. A single training run might consume 100–1000+ kWh. Without measurement, you're flying blind on optimization opportunities.

Key Takeaway: Measuring energy consumption across training, inference, and storage using hardware monitors and software tools is essential for understanding and reducing ML's environmental impact.