Course contentsShow
← Computer Science
Lesson 2794 of 2,87262. Machine Learning FundamentalsFree lesson

Label Quality and Acquisition

Understanding the cost and importance of obtaining high-quality labels for supervised learning tasks.

Label Quality and Acquisition

You'll learn: Why high-quality labels are expensive but essential, and how label acquisition strategies impact supervised learning success.

The Cost of Good Labels

In supervised learning, your model is only as good as the data it learns from—and obtaining accurate labels is often the most expensive part of the entire ML pipeline. Unlike raw data (images, text, sensor readings), which can sometimes be collected automatically, labels typically require human judgment, domain expertise, or expensive experiments.

Consider training a medical diagnosis classifier: each label requires a qualified radiologist to examine an X-ray and provide a diagnosis. At hundreds of dollars per hour, labeling thousands of examples becomes prohibitively expensive. Yet if those labels are wrong—misdiagnoses, inconsistent annotations, or ambiguous cases—your model will learn incorrect patterns and fail in production.

Quality vs Quantity Tradeoffs

Label quality encompasses:

  • Accuracy: Is the label correct?
  • Consistency: Do multiple annotators agree?
  • Coverage: Do labels represent all important cases?
  • Noise tolerance: How many errors can your training process handle?

Acquisition strategies balance cost and quality:

  • Expert labeling: Expensive but accurate (doctors, lawyers, specialists)
  • Crowdsourcing: Cheaper but noisier (Amazon Mechanical Turk)
  • Semi-automated: Model-assisted labeling where humans verify predictions
  • Active learning: Strategically choosing which examples to label for maximum impact

Think of it like building a recipe book: hiring professional chefs (experts) ensures accurate recipes but costs more; asking home cooks (crowdsourcing) is cheaper but yields inconsistent results. Your supervised learning model will inherit whichever quality you choose.

Key Takeaway: Label acquisition is typically the bottleneck in supervised learning—investing in high-quality labels early often yields better models than collecting massive amounts of noisy data cheaply.