Course contentsShow
← Machine Learning and Deep Learning
Lesson 506 of 3,53812. Model Validation and TuningFree lesson

The Hyperparameter Search Space

Define search spaces with continuous, discrete, and categorical hyperparameters.

The Hyperparameter Search Space

What you'll learn: You'll understand how to define the range and types of hyperparameters your model will explore during tuning.

What Is a Hyperparameter Search Space?

The search space is the complete set of possible values you allow each hyperparameter to take when tuning your model. Think of it like a treasure map: you define the boundaries of where to search, and the tuning algorithm explores within those boundaries to find the best combination.

Before you can tune hyperparameters (which you learned are model settings you control before training), you must define what values are even worth trying.

Three Types of Search Spaces

Continuous Hyperparameters

These can take any value within a range. Example: a learning rate between 0.001 and 0.1. You might search anywhere along that continuum (0.005, 0.0237, 0.089, etc.).

Discrete Hyperparameters

These are numeric but restricted to specific intervals or integers. Example: the number of trees in a random forest must be a whole number (50, 100, 150), not 73.5.

Categorical Hyperparameters

These are non-numeric choices from a fixed set. Example: activation functions like 'relu', 'tanh', or 'sigmoid'. There's no "in-between" option—you pick one from the menu.

Why This Matters

Defining your search space well means the difference between finding a great model quickly and wasting compute time exploring irrelevant values. Too narrow, and you might miss the optimal setting. Too wide, and you'll search forever without convergence.

Key Takeaway: The hyperparameter search space defines the boundaries (continuous ranges, discrete steps, or categorical options) within which your tuning algorithm explores to find optimal model settings.