Course contentsShow
← Machine Learning and Deep Learning
Lesson 2702 of 3,53858. Model Compression: Pruning and DistillationFree lesson

AutoML Frameworks and Practical NAS

Using tools like Auto-Keras, NNI, and Optuna for accessible NAS in real-world applications.

AutoML Frameworks and Practical NAS

What you'll learn: How production-ready AutoML tools make Neural Architecture Search accessible without implementing search algorithms from scratch.

Why Frameworks Matter

You've learned about NAS search strategies (RL, evolutionary, gradient-based) and performance estimation techniques. Implementing these from scratch is complex and time-consuming. AutoML frameworks package these algorithms into user-friendly APIs, letting you focus on your problem rather than NAS mechanics.

Popular AutoML Frameworks

Auto-Keras

An easy-to-use wrapper built on Keras/TensorFlow. It automates architecture search for common tasks (image classification, text classification) with minimal code. You specify input/output, and it handles the search space and strategy internally.

NNI (Neural Network Intelligence)

Microsoft's toolkit supporting multiple NAS algorithms (DARTS, ENAS, evolutionary methods). It offers fine-grained control over search spaces and strategies while providing experiment tracking and visualization. Great for researchers who need flexibility.

Optuna

A general hyperparameter optimization framework that extends naturally to NAS. You define the architecture as a search space of hyperparameters (number of layers, filter sizes, etc.), and Optuna's Bayesian optimization or tree-structured Parzen estimators find good configurations efficiently.

Practical Benefits

These frameworks handle boilerplate code: weight sharing schemes, performance estimation, distributed training across candidate architectures, and early stopping of poor performers. They also integrate with hardware-aware NAS—Auto-Keras and NNI can optimize for latency or memory constraints, not just accuracy.

Analogy: If NAS algorithms are recipes, these frameworks are meal-kit services—ingredients pre-measured, instructions simplified, letting you cook (search) without being a chef (NAS expert).

Key Takeaway: AutoML frameworks like Auto-Keras, NNI, and Optuna democratize NAS by providing production-ready implementations of search algorithms, making architecture optimization practical for real-world applications without deep NAS expertise.