Course contentsShow
Machine Learning and Deep Learning
Lesson 2732 of 3,53859. Distributed Training: Data ParallelismPro lesson

All-Gather and Reduce-Scatter Operations

The core collective communication primitives FSDP uses to gather full parameters and distribute gradients.

This lesson is for subscribers

You've completed the free preview. Subscribe to unlock every lesson in every course.