Course contentsShow
← Machine Learning and Deep Learning
Lesson 89 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Hypothesis Testing Framework

Formulating null and alternative hypotheses, p-values, and significance levels for statistical tests.

Hypothesis Testing Framework

What you'll learn: You'll learn how to set up and interpret a formal statistical test to decide whether observed data provides strong enough evidence to reject a baseline assumption.

The Core Idea

Think of hypothesis testing like a courtroom trial. We start by assuming innocence (our null hypothesis, often written as H₀), then ask: "Is the evidence strong enough to reject this assumption?"

In statistics, the null hypothesis (H₀) represents the status quo or "no effect" claim. For example: "This new ML model performs no better than random guessing" or "The mean accuracy is 50%."

The alternative hypothesis (H₁ or Hₐ) is what you're trying to prove: "The model does better than random" or "The mean accuracy is greater than 50%."

P-Values and Significance Levels

After collecting data, we calculate a p-value: the probability of seeing results as extreme as ours if the null hypothesis were true.

  • A small p-value (like 0.01) means: "Wow, if H₀ were true, this data would be really unlikely. Maybe H₀ is wrong!"
  • A large p-value (like 0.60) means: "This data is pretty consistent with H₀. No reason to reject it."

The significance level (α) is your decision threshold, commonly set at 0.05. If p-value < α, you reject H₀ and accept H₁. If p-value ≥ α, you fail to reject H₀ (note: you don't "accept" it, you simply lack evidence against it).

Analogy

The p-value is like asking: "If my friend really is honest (H₀), what are the chances they'd 'accidentally' guess my password on the first try?" If that chance is tiny, you doubt their honesty claim.

Key Takeaway: Hypothesis testing provides a structured framework to evaluate claims using data: formulate H₀ and H₁, compute a p-value from your sample, and compare it to your significance level α to make a principled decision.