Binary Encoding for Memory Efficiency
What you'll learn: How to represent categorical variables as binary digits to save memory compared to one-hot encoding.
The Problem with One-Hot Encoding
Remember one-hot encoding? If you have 100 categories, you create 100 new columns—one per category. That's a lot of memory! Binary encoding offers a smarter alternative by using the mathematical property of binary numbers to represent categories.
How Binary Encoding Works
Binary encoding converts each category into a binary number (base-2), then splits those digits into separate columns.
Think of it like this: Instead of giving each person at a party their own room (one-hot), you assign them binary ID badges. With just 4 binary digits (0s and 1s), you can uniquely identify 16 people using only 4 columns instead of 16!
The Math Behind It
- 10 categories → need log₂(10) ≈ 4 binary columns (vs. 10 one-hot columns)
- 100 categories → need log₂(100) ≈ 7 binary columns (vs. 100 one-hot columns)
- 1000 categories → need log₂(1000) ≈ 10 binary columns (vs. 1000 one-hot columns)
Example Walkthrough
Say you have categories: ['Red', 'Blue', 'Green', 'Yellow']
- Assign each an integer: Red=0, Blue=1, Green=2, Yellow=3
- Convert to binary: 0→00, 1→01, 2→10, 3→11
- Split into columns:
bit_0andbit_1
This gives you 2 columns instead of 4 with one-hot encoding!
When to Use Binary Encoding
Binary encoding shines with high-cardinality nominal variables (50+ categories) where one-hot encoding would explode your dataset's width. It's particularly useful when memory is constrained.
Key Takeaway: Binary encoding compresses categorical data by representing N categories with only log₂(N) columns, dramatically reducing memory usage compared to one-hot encoding's N columns.