Course contentsShow
← Machine Learning and Deep Learning
Lesson 425 of 3,53810. Feature EngineeringFree lesson

Binary Encoding for Memory Efficiency

Converting categories to binary representation, reducing dimensionality compared to one-hot encoding.

Binary Encoding for Memory Efficiency

What you'll learn: How to represent categorical variables as binary digits to save memory compared to one-hot encoding.

The Problem with One-Hot Encoding

Remember one-hot encoding? If you have 100 categories, you create 100 new columns—one per category. That's a lot of memory! Binary encoding offers a smarter alternative by using the mathematical property of binary numbers to represent categories.

How Binary Encoding Works

Binary encoding converts each category into a binary number (base-2), then splits those digits into separate columns.

Think of it like this: Instead of giving each person at a party their own room (one-hot), you assign them binary ID badges. With just 4 binary digits (0s and 1s), you can uniquely identify 16 people using only 4 columns instead of 16!

The Math Behind It

  • 10 categories → need log₂(10) ≈ 4 binary columns (vs. 10 one-hot columns)
  • 100 categories → need log₂(100) ≈ 7 binary columns (vs. 100 one-hot columns)
  • 1000 categories → need log₂(1000) ≈ 10 binary columns (vs. 1000 one-hot columns)

Example Walkthrough

Say you have categories: ['Red', 'Blue', 'Green', 'Yellow']

  1. Assign each an integer: Red=0, Blue=1, Green=2, Yellow=3
  2. Convert to binary: 0→00, 1→01, 2→10, 3→11
  3. Split into columns: bit_0 and bit_1

This gives you 2 columns instead of 4 with one-hot encoding!

When to Use Binary Encoding

Binary encoding shines with high-cardinality nominal variables (50+ categories) where one-hot encoding would explode your dataset's width. It's particularly useful when memory is constrained.

Key Takeaway: Binary encoding compresses categorical data by representing N categories with only log₂(N) columns, dramatically reducing memory usage compared to one-hot encoding's N columns.