XNOR-Net and Bitwise Operations
You'll learn how binary neural networks achieve extreme speedups using simple bitwise operations instead of floating-point arithmetic.
The Core Idea
In XNOR-Net, both weights and activations are binarized to {-1, +1}. The brilliant insight: when you represent -1 as bit 0 and +1 as bit 1, the standard convolution operation transforms into ultra-fast bitwise operations.
Here's the magic: multiplying two binary values (+1 or -1) is equivalent to an XNOR operation on their bit representations, followed by a popcount (counting the number of 1s).
Why XNOR?
Consider multiplying binarized values:
- (+1) × (+1) = +1 → bits (1, 1) → XNOR = 1
- (+1) × (-1) = -1 → bits (1, 0) → XNOR = 0
- (-1) × (+1) = -1 → bits (0, 1) → XNOR = 0
- (-1) × (-1) = +1 → bits (0, 0) → XNOR = 1
The XNOR result matches the sign of multiplication! For a full convolution, you XNOR corresponding bits, then use popcount to count matches, and convert: result = 2 × popcount - n, where n is the total number of bits.
The Performance Win
CPUs can perform XNOR on 64 bits simultaneously in a single instruction. Popcount is also hardware-accelerated. This means you replace expensive floating-point multiply-accumulate operations with bitwise operations that are ~58× faster and consume negligible power.
Key Takeaway: XNOR-Net exploits binary representations to replace convolution's floating-point multiplications with bitwise XNOR and popcount operations, achieving massive speedups with minimal hardware requirements.