Course contentsShow
← Machine Learning and Deep Learning
Lesson 2619 of 3,53857. Model Compression: QuantizationFree lesson

Quantization Impact on Model Size

Calculating memory savings: how moving from FP32 to INT8 reduces model size by 4× with concrete examples.

Quantization Impact on Model Size

What you'll learn: How switching from 32-bit floating point (FP32) to 8-bit integer (INT8) quantization reduces your model's storage footprint by roughly 4×.

The Memory Math

When you quantize a model from FP32 to INT8, you're shrinking each weight and activation from 32 bits down to 8 bits. The math is straightforward:

32 bits ÷ 8 bits = 4× compression

Every parameter that once required 4 bytes now fits in just 1 byte. For a model with millions or billions of parameters, this adds up dramatically.

Concrete Example

Imagine a small convolutional layer with 1 million parameters:

  • FP32 storage: 1,000,000 parameters × 4 bytes = 4 MB
  • INT8 storage: 1,000,000 parameters × 1 byte = 1 MB
  • Savings: 3 MB (75% reduction)

Now scale this to a modern language model like BERT-Base with ~110 million parameters:

  • FP32: 110M × 4 bytes ≈ 440 MB
  • INT8: 110M × 1 byte ≈ 110 MB
  • Savings: ~330 MB

For larger models (billions of parameters), this 4× reduction means the difference between fitting on device memory or not—critical for edge deployment on phones or IoT devices.

The Trade-off

The compression is near-perfect (exactly 4×), but remember: you're trading memory for a small accuracy cost. The discrete 256 values in INT8 can't perfectly represent the continuous range of FP32, introducing quantization error.

Key Takeaway: Quantizing from FP32 to INT8 delivers a consistent 4× model size reduction by using one-quarter the bits per parameter—transforming gigabyte-scale models into hundred-megabyte packages suitable for resource-constrained environments.