Course contentsShow
← AI Engineering
Lesson 189 of 1,8865. Advanced Prompting TechniquesFree lesson

Majority Voting and Answer Aggregation

Aggregate multiple LLM outputs using majority voting to identify the most likely correct answer.

Majority Voting and Answer Aggregation

What You'll Learn

You'll learn how to combine multiple LLM outputs to identify the most reliable answer by counting which response appears most frequently.

The Core Idea

When you use self-consistency with temperature sampling, you generate multiple reasoning paths for the same question. But now you have 5, 10, or more different answers—how do you pick the best one?

Majority voting is the simple, powerful solution: count how many times each answer appears, and choose the one that shows up most often. It's like polling a group of experts and going with the consensus answer.

Why This Works

LLMs can make random errors on individual attempts, but correct reasoning is more likely to converge on the same answer. If you generate 7 responses and 5 say "42" while 2 say "41", the answer "42" is probably correct—those outliers might reflect rare reasoning mistakes.

The Process

  1. Generate multiple completions (using temperature > 0 for diversity)
  2. Extract the final answer from each response
  3. Count occurrences of each unique answer
  4. Select the answer with the highest count

A Simple Analogy

Imagine asking 10 calculators to solve a math problem, but 2 have sticky buttons. Most will give the correct answer; the broken ones might not. You go with whatever answer the majority reports.

Ties and Edge Cases

If two answers tie for most votes, you have options:

  • Return both as equally likely
  • Use a tiebreaker rule (first occurrence, random selection)
  • Generate one more sample to break the tie

Key Takeaway: Majority voting aggregates multiple LLM outputs by selecting the answer that appears most frequently, leveraging the wisdom-of-crowds principle to filter out individual errors and improve accuracy.