Course contentsShow
← Machine Learning and Deep Learning
Lesson 2624 of 3,53857. Model Compression: QuantizationFree lesson

Uniform vs Non-Uniform Quantization

Equally spaced quantization levels vs adaptive binning strategies like k-means clustering for better distribution matching.

Uniform vs Non-Uniform Quantization

What you'll learn: You'll understand the difference between equally spaced quantization levels and adaptive strategies that better match your data's distribution.

The Core Idea

In quantization, you're mapping high-precision values (like 32-bit floats) to a smaller set of discrete levels. Uniform quantization spaces these levels evenly across your range—like marking a ruler with equally spaced tick marks. Non-uniform quantization adapts the spacing to where your values actually cluster—like putting more tick marks where you need finer measurements.

Uniform Quantization

This is what we've been using with symmetric and asymmetric quantization. You divide your range into equal-sized bins. If your weight values range from -1.0 to 1.0 and you quantize to 8 bits (256 levels), each bin has width 2.0/256 ≈ 0.0078.

Pros: Simple, hardware-friendly, fast computation
Cons: Wasteful if your data clusters unevenly

Non-Uniform Quantization

Instead of equal spacing, you place quantization levels where your data is dense. K-means clustering is a popular approach: you cluster your weight values into k groups, then use the cluster centroids as your quantization levels.

Analogy: Imagine parking spaces in a city. Uniform quantization spreads them evenly across all streets. Non-uniform quantization puts more spaces downtown where people actually go.

Pros: Better represents actual data distribution, can achieve lower error
Cons: More complex to compute, requires hardware support for lookup tables, adds overhead

When to Use Which

Use uniform quantization for standard deployment—it's supported everywhere. Consider non-uniform when you have extreme value clustering and can afford the implementation complexity.

Key Takeaway: Uniform quantization uses equally spaced levels for simplicity and hardware compatibility, while non-uniform quantization (like k-means-based binning) adapts to your data distribution for potentially better accuracy at the cost of increased complexity.