Course contentsShow
← Machine Learning and Deep Learning
Lesson 505 of 3,53812. Model Validation and TuningFree lesson

What Are Hyperparameters vs Parameters

Distinguish between learnable parameters and hyperparameters that control the learning process.

What Are Hyperparameters vs Parameters

What you'll learn: You'll understand the critical difference between parameters that a model learns during training and hyperparameters that you set before training begins.

The Core Distinction

Think of building a model like training an athlete. Parameters are the skills the athlete develops through practice—muscle memory, technique, reflexes. The model learns these directly from the data during training. In a neural network, parameters are the weights and biases that get adjusted with each training iteration.

Hyperparameters, on the other hand, are the training conditions you set up before practice begins—how many hours to train per day, what diet to follow, which exercises to focus on. These control how the learning happens but aren't learned from the data itself.

Concrete Examples

Parameters (learned from data):

  • Weights in a neural network
  • Coefficients in linear regression
  • Split thresholds in decision trees

Hyperparameters (you configure):

  • Learning rate
  • Number of training epochs
  • Number of trees in a random forest
  • K in K-Fold cross-validation
  • Regularization strength (like L1 penalty)
  • Batch size

Why This Matters

Parameters define what your model knows after training. Hyperparameters control the learning process itself. You can't learn optimal hyperparameters the same way you learn parameters—you must test different hyperparameter combinations (which you'll explore in upcoming lessons on hyperparameter tuning).

When someone says "tuning a model," they almost always mean adjusting hyperparameters to find the configuration that produces the best-performing parameters.

Key Takeaway: Parameters are learned by the model during training; hyperparameters are configuration settings you choose beforehand that control how that learning happens.