Course contentsShow
← Machine Learning and Deep Learning
Lesson 2622 of 3,53857. Model Compression: QuantizationFree lesson

Quantization Parameters: Scale and Zero-Point

Understanding the scale factor and zero-point that define the mapping between quantized and real values.

Quantization Parameters: Scale and Zero-Point

You'll learn: How scale and zero-point mathematically define the conversion between floating-point and quantized integer values.

The Core Idea

When you quantize a neural network, you're mapping continuous floating-point numbers to discrete integers. But how does your model know which float corresponds to which integer? That's where scale and zero-point come in—they're the two parameters that define this mapping relationship.

Think of it like converting temperatures: to go from Celsius to Fahrenheit, you need a scaling factor (multiply by 9/5) and an offset (add 32). Similarly, quantization needs these parameters to translate between number systems.

The Mathematical Relationship

The conversion follows this formula:

quantized_value = round(real_value / scale) + zero_point

And to go back (dequantization):

real_value = (quantized_value - zero_point) × scale

Scale determines the step size between quantized levels. A smaller scale means finer granularity but smaller range. If your floats range from 0.0 to 25.5 and you're using 8-bit integers (0-255), your scale would be 0.1.

Zero-point maps a specific integer to the real value 0.0. In symmetric quantization (which you've already learned), zero-point is typically 0, making the mapping symmetric around zero. In asymmetric quantization, zero-point can be any integer, allowing better coverage of asymmetric ranges like [0, 100].

Example

If scale = 0.5 and zero_point = 128, then:

  • Real value 0.0 → quantized 128
  • Real value 10.0 → quantized 148
  • Real value -10.0 → quantized 108

Key Takeaway: Scale and zero-point are the essential parameters that enable lossless conversion formulas between floating-point and quantized integer representations, with scale controlling granularity and zero-point establishing the anchor for the mapping.