Course contentsShow
← Machine Learning and Deep Learning
Lesson 2657 of 3,53857. Model Compression: QuantizationFree lesson

XNOR-Net and Bitwise Operations

Exploiting binary operations for ultra-fast inference using bitwise XNOR and popcount operations.

XNOR-Net and Bitwise Operations

You'll learn how binary neural networks achieve extreme speedups using simple bitwise operations instead of floating-point arithmetic.

The Core Idea

In XNOR-Net, both weights and activations are binarized to {-1, +1}. The brilliant insight: when you represent -1 as bit 0 and +1 as bit 1, the standard convolution operation transforms into ultra-fast bitwise operations.

Here's the magic: multiplying two binary values (+1 or -1) is equivalent to an XNOR operation on their bit representations, followed by a popcount (counting the number of 1s).

Why XNOR?

Consider multiplying binarized values:

  • (+1) × (+1) = +1 → bits (1, 1) → XNOR = 1
  • (+1) × (-1) = -1 → bits (1, 0) → XNOR = 0
  • (-1) × (+1) = -1 → bits (0, 1) → XNOR = 0
  • (-1) × (-1) = +1 → bits (0, 0) → XNOR = 1

The XNOR result matches the sign of multiplication! For a full convolution, you XNOR corresponding bits, then use popcount to count matches, and convert: result = 2 × popcount - n, where n is the total number of bits.

The Performance Win

CPUs can perform XNOR on 64 bits simultaneously in a single instruction. Popcount is also hardware-accelerated. This means you replace expensive floating-point multiply-accumulate operations with bitwise operations that are ~58× faster and consume negligible power.

Key Takeaway: XNOR-Net exploits binary representations to replace convolution's floating-point multiplications with bitwise XNOR and popcount operations, achieving massive speedups with minimal hardware requirements.