Course contentsShow
← Machine Learning and Deep Learning
Lesson 3470 of 3,53876. Ethics and Societal ImpactFree lesson

Data Center Energy and Cooling Requirements

Explore the infrastructure costs of ML, including electricity for compute and cooling systems.

Data Center Energy and Cooling Requirements

What you'll learn: How training large ML models requires massive data center infrastructure, driving both electricity consumption and cooling system demands.

The Hidden Infrastructure Cost

When you train a large neural network, GPUs don't just consume electricity for computation—they generate enormous amounts of heat. This heat must be continuously removed, or your hardware will throttle performance or fail entirely. Data centers housing ML infrastructure therefore face a dual energy burden: powering the compute and powering the cooling systems.

The Cooling Challenge

Think of a data center like a high-performance race car: the engine (GPUs) produces tremendous power, but without a sophisticated cooling system (radiators, fans), it would overheat in minutes. Modern data centers use:

  • Air cooling systems: HVAC units that circulate cooled air, consuming 30-50% as much power as the compute itself
  • Liquid cooling: More efficient systems that circulate coolant directly to hot components
  • Chillers and cooling towers: Industrial equipment that dissipates heat into the environment

A typical GPU cluster at full utilization might draw 10 kW per rack. For every 10 kW of compute power, you might need an additional 3-5 kW just for cooling—a metric called Power Usage Effectiveness (PUE).

The Scale Problem

Training a large language model might use hundreds or thousands of GPUs simultaneously for weeks. If your cluster draws 2 MW for compute, you could need an additional 600 kW-1 MW just to keep it cool. This infrastructure cost is often invisible in ML carbon footprint calculations, yet it's substantial and geographically dependent (cooling costs more in hot climates).

Key Takeaway: The energy cost of ML extends far beyond GPU watts—data center cooling systems can add 30-100% overhead, making infrastructure efficiency a critical ethical and practical consideration in large-scale ML deployment.