Course contentsShow
← Machine Learning and Deep Learning
Lesson 2625 of 3,53857. Model Compression: QuantizationFree lesson

The Quantization Equation and Dequantization

Mathematical formula for converting between real and quantized values, and when dequantization is necessary.

The Quantization Equation and Dequantization

You'll learn the mathematical formula that bridges the gap between real-valued neural network weights and their compressed integer representations.

The Core Mathematical Relationship

When we quantize a floating-point value, we're essentially mapping it to a discrete integer using a linear transformation. The quantization equation is:

quantized_value = round(real_value / scale) + zero_point

Where:

  • real_value is your original floating-point number (e.g., a weight of 0.743)
  • scale determines the step size between quantized levels
  • zero_point shifts the range (used in asymmetric quantization)
  • round() converts to the nearest integer

Analogy: Think of a thermometer. The real temperature (continuous) gets mapped to discrete markings on the scale. The scale factor is like the spacing between degree marks, and zero_point is where you place the zero marker.

Going Backwards: Dequantization

To recover an approximate floating-point value from a quantized integer:

real_value ≈ (quantized_value - zero_point) × scale

Notice the ≈ symbol—you lose precision due to rounding, so dequantization doesn't perfectly reconstruct the original value.

When Is Dequantization Necessary?

You need dequantization when:

  • Mixing precision levels: Combining quantized layers with full-precision operations
  • Inference debugging: Inspecting intermediate values in human-readable form
  • Dynamic quantization: Converting back to float32 for certain operations that don't support integer arithmetic

Many optimized inference engines keep values in quantized form throughout computation, avoiding dequantization overhead entirely.

Key Takeaway: The quantization equation q = round(r/s) + z and its inverse r ≈ (q-z)×s form the mathematical foundation of model compression, trading perfect precision for massive size and speed gains.