Systems of Linear Equations

What you'll learn: How to represent multiple linear equations as a single matrix equation and determine when solutions exist.

The Core Idea

Imagine you're trying to find the recipe for a mystery smoothie by tasting three different batches. Each batch uses different amounts of strawberries, bananas, and protein powder, but you know the total calories. You need to solve for the unknowns (ingredient amounts) using multiple clues (equations).

In mathematics, when you have multiple equations with multiple unknowns, you have a system of linear equations. Instead of writing them separately, we can represent the entire system compactly as:

Ax = b

Where:

• A is a matrix containing all the coefficients
• x is a vector of unknowns you're solving for
• b is a vector of results

For example, these two equations:

2x₁ + 3x₂ = 8
1x₁ + 4x₂ = 9

Become:

A = [[2, 3],    x = [x₁],    b = [8]
     [1, 4]]         [x₂]         [9]

When Do Solutions Exist?

Not every system has a solution! Three scenarios:

1. One unique solution – equations intersect at exactly one point
2. Infinite solutions – equations describe the same line/plane
3. No solution – equations are parallel, never intersect

The key is matrix rank: the number of independent equations. If rank(A) equals the number of unknowns and rank(A) = rank([A|b]), a unique solution exists. If rank is less, you might have infinite solutions or none.

Key Takeaway: Systems of linear equations Ax=b compress multiple equations into one matrix equation, and whether solutions exist depends on the rank of matrix A — a fundamental tool throughout machine learning for solving parameter optimization problems.