Course contentsShow
Machine Learning and Deep Learning
Lesson 2745 of 3,53859. Distributed Training: Data ParallelismPro lesson

ZeRO Stage 2: Gradient Partitioning

Extending ZeRO-1 by also partitioning gradients across devices, further reducing memory footprint.

This lesson is for subscribers

You've completed the free preview. Subscribe to unlock every lesson in every course.