Course contentsShow
← Machine Learning and Deep Learning
Lesson 1728 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

4-bit NormalFloat (NF4) Quantization

Understanding NF4, a data type optimized for normally distributed weights, used as QLoRA's quantization format.

4-bit NormalFloat (NF4) Quantization

What you'll learn: How NF4 uses just 4 bits per weight by exploiting the normal distribution pattern found in neural network weights.

The Core Idea

Standard quantization methods (like uniform 4-bit) divide the numerical range into equally-spaced buckets. But neural network weights aren't uniformly distributed—they follow a normal (Gaussian) distribution, clustering heavily around zero with fewer values at the extremes.

NF4 is a specialized data type designed specifically for this reality. Instead of wasting precious bits on rarely-used extreme values, NF4 allocates more quantization levels to the dense central region where most weights live.

How NF4 Works

Think of it like parking spaces in a city: you don't put equal parking everywhere—you concentrate spaces where people actually go. NF4 does this by:

  1. Assuming weights are normally distributed (zero-centered)
  2. Placing quantization bins closer together near zero (where most weights cluster)
  3. Spacing bins farther apart at the tails (where few weights exist)

This gives you better precision where it matters most, even with only 16 possible values (2^4 = 16).

Why QLoRA Uses NF4

QLoRA specifically chose NF4 because:

  • Pretrained LLM weights empirically follow normal distributions
  • You get better accuracy than uniform 4-bit quantization
  • It still uses only 4 bits per parameter (75% memory reduction vs FP16)
  • The information-theoretically optimal encoding for normally distributed data

Key Takeaway: NF4 is a 4-bit quantization format optimized for normally distributed weights, achieving better accuracy than uniform quantization by concentrating precision where weight values are most dense—making it ideal for QLoRA's memory-efficient fine-tuning.