Course contentsShow
← Machine Learning and Deep Learning
Lesson 2701 of 3,53858. Model Compression: Pruning and DistillationFree lesson

Hardware-Aware NAS

Incorporating latency, energy, and memory constraints into architecture search objectives.

Hardware-Aware NAS

What you'll learn: How to make Neural Architecture Search consider real-world deployment constraints like latency, energy consumption, and memory footprint—not just accuracy.

The Problem with Accuracy-Only NAS

Traditional NAS optimizes for a single metric: validation accuracy. But the best-performing architecture might be too slow for real-time inference, drain a mobile device's battery in minutes, or exceed available RAM. A 99% accurate model is useless if it can't run on your target hardware.

Incorporating Hardware Constraints

Hardware-Aware NAS extends the search objective to balance accuracy with practical deployment metrics:

  • Latency: How fast the model runs (milliseconds per inference)
  • Energy: Power consumption during inference
  • Memory: RAM required for weights and activations

Instead of maximizing accuracy alone, you optimize a multi-objective function:

Objective = α × Accuracy - β × Latency - γ × Energy - δ × Memory

Or you might use Pareto optimization to find architectures that represent optimal trade-offs (e.g., "best accuracy for ≤50ms latency").

Measuring Hardware Metrics

You can't just count FLOPs—real hardware performance depends on:

  • Actual profiling: Run candidate architectures on target devices (mobile GPU, edge TPU, etc.)
  • Lookup tables: Pre-compute costs for common operations
  • Predictor models: Train ML models to estimate latency/energy from architecture descriptions

Practical Strategy

Hardware-Aware NAS typically measures these constraints during the performance estimation phase, rejecting or penalizing architectures that violate your deployment budget before expensive full training.

Key Takeaway: Hardware-Aware NAS turns architecture search into a constrained optimization problem, ensuring discovered models actually work on your target deployment platform—not just in academic benchmarks.