Parameters vs Hyperparameters
What you'll learn: You'll understand the crucial difference between parameters that a model learns during training and hyperparameters that you set before training begins.
The Core Distinction
In machine learning, parameters are the internal values that the model learns from your data during training. These are what the optimization algorithms (like gradient descent) adjust to minimize error.
In linear regression (y = mx + b), the slope (m) and intercept (b) are parameters. The model figures these out by analyzing your training data. You don't set them manually—the training process discovers the best values.
Hyperparameters, on the other hand, are configuration settings that you must choose before training starts. These control how the learning process itself behaves. The model doesn't learn these from data; you (or automated tuning methods like GridSearchCV) set them.
A Simple Analogy
Think of baking a cake:
- Parameters are like the final texture and taste that emerge from the baking process—they develop automatically based on what happens in the oven
- Hyperparameters are like the oven temperature and baking time you set beforehand—you control these directly
Examples in Context
In linear regression so far, you've mostly worked with parameters (weights and bias). But remember when you used StandardScaler? The decision to scale features is a hyperparameter choice. When you split data with train_test_split, the test_size value is a hyperparameter. With GridSearchCV, you're literally searching for the best hyperparameter values.
Key Takeaway: Parameters are learned automatically from data during training (like weights in linear regression), while hyperparameters are settings you choose before training that control the learning process itself.