BiteSizedChunks.comLearn one small thing at a time.

Course contentsShow

Machine Learning and Deep Learning

187What is Linear Regression?
188The Simple Linear Model: y = mx + b
189Parameters vs Hyperparameters
190Residuals and Prediction Errors
191The Mean Squared Error Loss Function
192The Least Squares Criterion
193The Closed-Form Solution (Normal Equation)
194Implementing Simple Linear Regression from Scratch
195Making Predictions with a Fitted Model
196Coefficient of Determination (R²)
197Assumptions of Simple Linear Regression
198Simple Linear Regression with Scikit-learn
199From Simple to Multiple Linear Regression
200Matrix Formulation of Multiple Linear Regression
201The Normal Equation Derivation
202Computing the Normal Equation in NumPy
203Interpreting Coefficients in Multiple Regression
204Multicollinearity and Its Effects
205Feature Scaling for Multiple Regression
206Polynomial and Interaction Features
207Evaluating Multiple Regression: R² and Adjusted R²
208Implementing Multiple Linear Regression in Scikit-learn
209From Analytical to Iterative: Why Gradient Descent?
210The Cost Function Landscape for Linear Regression
211The Gradient: Direction of Steepest Ascent
212Computing the Gradient of MSE
213The Gradient Descent Update Rule
214Batch Gradient Descent: Full Dataset Updates
215Learning Rate Selection and Convergence
216Stochastic Gradient Descent: Single-Sample Updates
217Mini-Batch Gradient Descent: The Practical Middle Ground
218Convergence Criteria and Stopping Conditions
219Feature Scaling for Gradient Descent
220Implementing Gradient Descent from Scratch
221The Problem of Overfitting in Linear Regression
222Bias-Variance Tradeoff in Regression
223Introduction to Regularization
224L2 Regularization and Ridge Regression
225Ridge Regression: Mathematical Formulation
226Ridge Regression: Closed-Form Solution
227L1 Regularization and Lasso Regression
228Lasso vs Ridge: Geometric Intuition
229Elastic Net: Combining L1 and L2
230Choosing the Regularization Parameter
231Feature Scaling for Regularized Regression
232Regularization Paths
233Implementing Ridge and Lasso in Scikit-learn
234When to Use Each Regularization Method

Machine Learning and Deep Learning

187What is Linear Regression?
188The Simple Linear Model: y = mx + b
189Parameters vs Hyperparameters
190Residuals and Prediction Errors
191The Mean Squared Error Loss Function
192The Least Squares Criterion
193The Closed-Form Solution (Normal Equation)
194Implementing Simple Linear Regression from Scratch
195Making Predictions with a Fitted Model
196Coefficient of Determination (R²)
197Assumptions of Simple Linear Regression
198Simple Linear Regression with Scikit-learn
199From Simple to Multiple Linear Regression
200Matrix Formulation of Multiple Linear Regression
201The Normal Equation Derivation
202Computing the Normal Equation in NumPy
203Interpreting Coefficients in Multiple Regression
204Multicollinearity and Its Effects
205Feature Scaling for Multiple Regression
206Polynomial and Interaction Features
207Evaluating Multiple Regression: R² and Adjusted R²
208Implementing Multiple Linear Regression in Scikit-learn
209From Analytical to Iterative: Why Gradient Descent?
210The Cost Function Landscape for Linear Regression
211The Gradient: Direction of Steepest Ascent
212Computing the Gradient of MSE
213The Gradient Descent Update Rule
214Batch Gradient Descent: Full Dataset Updates
215Learning Rate Selection and Convergence
216Stochastic Gradient Descent: Single-Sample Updates
217Mini-Batch Gradient Descent: The Practical Middle Ground
218Convergence Criteria and Stopping Conditions
219Feature Scaling for Gradient Descent
220Implementing Gradient Descent from Scratch
221The Problem of Overfitting in Linear Regression
222Bias-Variance Tradeoff in Regression
223Introduction to Regularization
224L2 Regularization and Ridge Regression
225Ridge Regression: Mathematical Formulation
226Ridge Regression: Closed-Form Solution
227L1 Regularization and Lasso Regression
228Lasso vs Ridge: Geometric Intuition
229Elastic Net: Combining L1 and L2
230Choosing the Regularization Parameter
231Feature Scaling for Regularized Regression
232Regularization Paths
233Implementing Ridge and Lasso in Scikit-learn
234When to Use Each Regularization Method

← Machine Learning and Deep Learning

Lesson 213 of 3,538·3. Linear RegressionPro lesson

The Gradient Descent Update Rule

Learn the iterative parameter update formula and understand how learning rate controls step size.

This lesson is for subscribers

You've completed the free preview. Subscribe to unlock every lesson in every course.

See pricing Back to course