Bernoulli Naive Bayes for Binary Features
What you'll learn: How Bernoulli Naive Bayes handles binary (yes/no) features differently from the count-based multinomial variant you just learned.
The Binary Feature Problem
While Multinomial Naive Bayes counts how many times features occur (word frequencies in documents), Bernoulli Naive Bayes focuses on whether features are present or absent. Think of it as switching from "How many times does 'sale' appear?" to "Does 'sale' appear at all?"
How Bernoulli Distributions Work
A Bernoulli distribution models binary outcomes—like coin flips. For each feature, the model learns:
- P(feature is present | class)
- P(feature is absent | class)
Crucially, absence is explicitly modeled. If "urgent" doesn't appear in an email, that absence contributes evidence. Multinomial NB would simply see a zero count and move on; Bernoulli NB treats non-occurrence as informative.
When Bernoulli Outperforms Multinomial
Use Bernoulli Naive Bayes when:
- Features are naturally binary (checkbox selections, presence/absence of keywords)
- You care about what's missing as much as what's present
- Document length varies wildly—Bernoulli ignores repetition
For example, spam detection often benefits from Bernoulli NB because the absence of professional language matters just as much as the presence of spam triggers. If "company footer" is missing, that's a spam signal.
Multinomial works better when frequency matters—like sentiment analysis where repeating "excellent" three times is stronger than once.
Practical Distinction
Imagine classifying short text messages. Multinomial counts: "free" appears 2 times. Bernoulli asks: "free" present? Yes (1) or No (0). The second representation ignores how many times, focusing purely on boolean presence.
Key Takeaway: Bernoulli Naive Bayes models features as binary (present/absent) and explicitly uses absence as evidence, making it ideal when feature presence—not frequency—determines class membership.