Course contentsShow
← Data Science
Lesson 1584 of 2,14538. Bayesian Data AnalysisFree lesson

Expected Loss and Decision Making

Use expected loss to quantify the cost of choosing the wrong variant and inform stopping decisions.

Expected Loss and Decision Making

You'll learn how to use expected loss to quantify the financial or practical cost of choosing the wrong variant and decide when to stop your A/B test.

What is Expected Loss?

Knowing that variant B has a 70% probability of being best is useful, but it doesn't tell you how much worse variant A might be if you choose it instead. Expected loss answers: "If I pick this variant and it's wrong, how much will I lose on average?"

For each variant, you calculate the expected loss by looking at all possible outcomes (based on your posterior distribution) and computing the average difference between that variant and the best possible choice.

How It Informs Decisions

Imagine you're testing two checkout button colors. Variant A has 65% probability of being best, but if B is actually better, you'd only lose 50/dayinrevenue.VariantBhas3550/day in revenue. Variant B has 35% probability of being best, but if A is better, you'd lose 500/day.

The expected losses might be:

  • Expected loss of choosing A: 0.35 × 500=500 = 175/day
  • Expected loss of choosing B: 0.65 × 50=50 = 32.50/day

Even though A has higher probability of being best, choosing B minimizes your expected loss—it's the safer, more economical decision.

Stopping Rules

You can set a threshold like "stop the test when expected loss < $10/day for the best variant." This gives you a concrete, business-aligned criterion instead of arbitrary probability thresholds.

Key Takeaway: Expected loss translates Bayesian probabilities into real costs, helping you make economically rational decisions about when to stop testing and which variant to deploy.