Course contentsShow
← Machine Learning and Deep Learning
Lesson 335 of 3,5387. Instance-Based and Probabilistic ClassifiersFree lesson

Training Naive Bayes: Parameter Estimation

Estimate class priors and conditional probabilities from training data using maximum likelihood.

Training Naive Bayes: Parameter Estimation

What you'll learn: How Naive Bayes classifiers learn from training data by estimating class priors and conditional probabilities using maximum likelihood.

The Core Idea

Training a Naive Bayes classifier means calculating two key sets of parameters from your training data:

  1. Class priors P(class): How often each class appears in your training set
  2. Conditional probabilities P(feature|class): The likelihood of each feature value given a specific class

These parameters are estimated using maximum likelihood estimation — essentially counting occurrences and computing frequencies.

How Parameter Estimation Works

Think of training as building a probability catalog. For class priors, you simply count:

P(class = spam) = (number of spam emails) / (total emails)

For conditional probabilities, you calculate feature distributions within each class:

  • Gaussian Naive Bayes: Compute the mean and variance of each feature for each class
  • Multinomial Naive Bayes: Count how often each word appears in documents of each class
  • Bernoulli Naive Bayes: Calculate the proportion of documents where each feature is present/absent per class

Why This Matters

Unlike iterative algorithms (gradient descent), Naive Bayes training is a single-pass computation. You scan through your training data once, accumulate counts and statistics, then compute your parameters. This makes training extremely fast, even on large datasets.

Remember that Laplace smoothing (from the previous lesson) is applied during this estimation phase to handle features that never appear with certain classes in the training data.

Key Takeaway: Training Naive Bayes involves estimating class priors by counting class frequencies and conditional probabilities by computing feature statistics within each class using maximum likelihood — a straightforward, one-pass calculation that makes it one of the fastest classifiers to train.