Course contentsShow
← Machine Learning and Deep Learning
Lesson 334 of 3,5387. Instance-Based and Probabilistic ClassifiersFree lesson

Laplace Smoothing for Zero Probabilities

Prevent zero probability issues by adding pseudocounts to handle unseen feature-class combinations.

Laplace Smoothing for Zero Probabilities

What You'll Learn

You'll learn how to prevent Naive Bayes classifiers from crashing when they encounter feature values they've never seen during training.

The Zero Probability Problem

When using Naive Bayes classifiers, you multiply probabilities of individual features together. But what happens if your training data never showed a particular word in spam emails, and then you encounter that word when classifying a new email?

The probability becomes zero. And when you multiply by zero, your entire posterior probability becomes zero—even if all other features strongly suggest "spam." One unseen feature value destroys your prediction.

How Laplace Smoothing Works

Laplace smoothing (also called additive smoothing) adds a small "pseudocount" to every possible feature-class combination, even those you've never observed.

Instead of counting:

  • "Word 'lottery' appeared 10 times in spam emails"

You pretend:

  • "Word 'lottery' appeared 10 + α times in spam emails"

The constant α (alpha) is typically 1, giving every feature a minimum count of 1. This ensures no probability is ever exactly zero.

Analogy: Imagine a coin you've never flipped. Without smoothing, you'd say "probability of heads = 0/0 = undefined." With Laplace smoothing (α=1), you assume you've already seen 1 head and 1 tail, so P(heads) = 1/2—a reasonable starting guess.

Why It Matters

Laplace smoothing makes Naive Bayes robust to unseen data. It prevents mathematical errors while keeping probabilities small but non-zero for rare events. The smoothing parameter α controls how much "faith" you place in unseen combinations versus observed data.

Key Takeaway: Laplace smoothing adds pseudocounts (typically α=1) to all feature-class combinations, preventing zero probabilities that would otherwise break Naive Bayes predictions when encountering unseen features.