Quantization Parameters: Scale and Zero-Point
You'll learn: How scale and zero-point mathematically define the conversion between floating-point and quantized integer values.
The Core Idea
When you quantize a neural network, you're mapping continuous floating-point numbers to discrete integers. But how does your model know which float corresponds to which integer? That's where scale and zero-point come in—they're the two parameters that define this mapping relationship.
Think of it like converting temperatures: to go from Celsius to Fahrenheit, you need a scaling factor (multiply by 9/5) and an offset (add 32). Similarly, quantization needs these parameters to translate between number systems.
The Mathematical Relationship
The conversion follows this formula:
quantized_value = round(real_value / scale) + zero_point
And to go back (dequantization):
real_value = (quantized_value - zero_point) × scale
Scale determines the step size between quantized levels. A smaller scale means finer granularity but smaller range. If your floats range from 0.0 to 25.5 and you're using 8-bit integers (0-255), your scale would be 0.1.
Zero-point maps a specific integer to the real value 0.0. In symmetric quantization (which you've already learned), zero-point is typically 0, making the mapping symmetric around zero. In asymmetric quantization, zero-point can be any integer, allowing better coverage of asymmetric ranges like [0, 100].
Example
If scale = 0.5 and zero_point = 128, then:
- Real value 0.0 → quantized 128
- Real value 10.0 → quantized 148
- Real value -10.0 → quantized 108
Key Takeaway: Scale and zero-point are the essential parameters that enable lossless conversion formulas between floating-point and quantized integer representations, with scale controlling granularity and zero-point establishing the anchor for the mapping.