Course contentsShow
← Machine Learning and Deep Learning
Lesson 685 of 3,53817. Optimization for Deep LearningFree lesson

Batch Size Effects on Training

How batch size impacts convergence speed, generalization, memory usage, and gradient noise.

Batch Size Effects on Training

What you'll learn: How the number of samples you process at once fundamentally affects your neural network's learning behavior, memory usage, and final performance.

The Core Idea

When training with mini-batch gradient descent, you choose how many samples to process before updating weights. This seemingly simple choice—your batch size—has profound ripple effects throughout training.

Think of batch size like sample size in a survey: a small sample (small batch) gives noisy but frequent feedback, while a large sample (large batch) provides stable but infrequent updates.

Four Key Effects

1. Convergence Speed

Small batches (8-32): Noisy gradients lead to more erratic updates, but you update weights more frequently per epoch. Often reach "good enough" solutions faster in wall-clock time.

Large batches (256-1024+): Smoother, more stable gradient estimates. Each update is more accurate, but you make fewer updates per epoch.

2. Generalization

Small batches introduce gradient noise that acts like a regularizer, helping models escape sharp minima and find flatter regions that generalize better. Large batches can converge to sharper minima with worse test performance (though this is actively researched).

3. Memory Usage

Larger batches require proportionally more GPU memory to store activations for all samples during the forward and backward passes.

4. Gradient Noise

Small batches = high variance gradients (noisier signal)
Large batches = low variance gradients (cleaner but less exploratory)

Practical Trade-offs

Common sweet spot: 32-256 samples, balancing memory efficiency, hardware utilization, and generalization benefits.

Key Takeaway: Batch size is a crucial hyperparameter that trades off gradient noise (good for exploration and generalization) against computational efficiency and stability—smaller batches often generalize better despite noisier training.