Perceptron Learning Rule: Training a Single Neuron
What you'll learn: How a perceptron adjusts its weights based on mistakes to learn simple patterns.
The Core Idea
The perceptron learning rule is an iterative algorithm that teaches a single neuron to classify data correctly. Think of it like teaching a child to sort apples from oranges: when they make a mistake, you gently correct them, and over time they learn the distinction.
Here's how it works:
- Make a prediction using current weights
- Compare to the true label — did we get it right?
- Update weights only if wrong — adjust based on the error
- Repeat until all training examples are classified correctly (or a maximum number of iterations)
The Update Rule
When the perceptron makes an error, it updates each weight using:
w_new = w_old + learning_rate × error × input_value
Where:
- error = (true_label - predicted_label)
- learning_rate controls the step size (typically a small value like 0.1)
If the prediction is correct, no update happens. If wrong, the weights shift in the direction that would have produced the correct answer.
Why This Works for Linearly Separable Data
The perceptron learning rule is guaranteed to converge if the data can be separated by a straight line (or hyperplane in higher dimensions). Each mistake pushes the decision boundary closer to correctly dividing the classes. However, if the data isn't linearly separable (like XOR), the perceptron will never settle on a solution.
Key Takeaway: The perceptron learning rule incrementally corrects weights based on classification errors, guaranteeing convergence for linearly separable patterns—a foundational example of how neural networks learn from mistakes.