Federated Learning vs Centralized Training
What you'll learn: How federated learning and centralized training differ in architecture, privacy protection, and computational costs.
Core Architectural Differences
In centralized training, all data flows to a single location—a central server or cloud datacenter—where a model trains on the aggregated dataset. Think of it like students mailing their homework to one teacher who grades everything in their office.
Federated learning flips this model: the training algorithm travels to where the data lives. Each participant (client device, hospital, organization) trains locally on their own data, then only shares model updates (gradients or weights) back to a central coordinator. The teacher visits each student's home, observes their work locally, and only takes notes back to combine insights.
Privacy Benefits
Centralized training requires collecting raw data in one place, creating a single point of vulnerability. If that central server is compromised, all data is exposed.
Federated learning keeps raw data decentralized. Since only model updates leave each client, sensitive information (medical records, personal messages) never travels across the network. This provides data minimization—a core privacy principle—though updates themselves can still leak information (remember gradient-based privacy attacks from earlier lessons).
Computational Trade-offs
Centralized training offers efficiency: faster convergence with synchronized updates, easier debugging, and consistent hardware. Federated learning faces heterogeneity challenges: devices vary in compute power, network speed, and data distribution. Training takes longer due to communication rounds, stragglers (slow clients), and non-IID (non-identical) data across participants.
However, federated learning scales naturally—thousands of edge devices contribute without bottlenecking a central server.
Key Takeaway: Centralized training prioritizes speed and simplicity by pooling data centrally; federated learning prioritizes privacy by keeping data local, trading off communication overhead and heterogeneity challenges for stronger data protection.