Right-Sizing Compute Resources
What you'll learn: How to match compute instances to your actual workload needs, cutting costs without sacrificing performance.
The Core Problem
After understanding what drives infrastructure costs (lesson 1209), the next step is fixing the most common waste: running machines that are too powerful for what they're actually doing. Right-sizing means choosing instance types that match your workload's CPU, memory, GPU, and throughput requirements—no more, no less.
The Goldilocks Principle
Think of it like buying a car. You wouldn't buy an 18-wheeler to commute alone, nor would you use a sedan to haul freight. Yet many teams provision GPU-heavy instances for simple preprocessing tasks or run lightweight APIs on oversized servers "just in case."
How to Right-Size
Start with measurement. Before changing anything, track actual resource usage:
- CPU utilization (target: 60-80% during typical load)
- Memory consumption (avoid constant swapping)
- GPU usage (expensive when idle)
- Network throughput
Match instance families to workload patterns:
- CPU-bound preprocessing? Compute-optimized instances (c-series)
- Memory-intensive vector operations? Memory-optimized (r-series)
- Model inference? GPU instances—but only where needed
- Background batch jobs? Spot instances or smaller nodes
Avoid over-provisioning from fear. That "what if we get a spike?" anxiety leads to waste. Use autoscaling (lessons 1125-1128) instead of permanently large instances.
Common Wins
- Moving non-GPU preprocessing off GPU nodes (50-70% cost reduction)
- Downsizing idle development environments
- Using smaller instances for rate-limited API calls
- Splitting workloads across appropriate instance types
Key Takeaway: Right-sizing eliminates waste by matching instance capabilities to actual workload demands—measure usage, choose appropriate instance types, and let autoscaling handle peaks instead of permanently over-provisioning.