Precision Types: FP32, FP16, INT8, INT4
What you'll learn: How different numerical precision formats trade memory and speed for accuracy when storing model weights.
The Core Concept
Model weights are just numbers, and computers can represent numbers with varying levels of precision. FP32 (32-bit floating point) is the standard "full precision" format—it uses 32 bits per number, offering high accuracy but consuming more memory. FP16 (16-bit floating point) cuts memory in half while maintaining decent range and precision. INT8 (8-bit integer) uses just 8 bits and requires careful calibration to map continuous values into discrete integers. INT4 (4-bit integer) is the most aggressive, using only 4 bits per weight.
Think of it like photograph quality: FP32 is a RAW file with every detail, FP16 is a high-quality JPEG, INT8 is a compressed JPEG, and INT4 is a heavily compressed thumbnail. Each step down reduces file size dramatically, but you lose some visual fidelity.
Practical Impact
Switching from FP32 to FP16 reduces model size by 50% with minimal accuracy loss—most modern LLMs train in FP16. Quantizing to INT8 cuts size by 75% (4× smaller than FP32) and often speeds up inference, with accuracy typically degrading only slightly if done carefully. INT4 achieves an 87.5% reduction (8× smaller), enabling very large models to fit in memory, though accuracy impact becomes more noticeable.
The choice depends on your constraints: if you're memory-limited but need good accuracy, INT8 is often the sweet spot. If you're running on edge devices with tight constraints, INT4 might be necessary despite higher accuracy loss.
Key Takeaway: Lower precision formats (FP16, INT8, INT4) dramatically reduce model size and increase inference speed at the cost of some numerical accuracy—quantization is essential for efficient deployment of large models.