Cross-Device vs Cross-Silo Federated Learning
What you'll learn: You'll understand the two major deployment patterns for federated learning and how they differ in scale, reliability, and privacy requirements.
The Two Federated Paradigms
Federated learning splits into two architectural families based on who holds the data and how many participants join training.
Cross-Device Federated Learning
This is the mobile scenario you've seen with smartphones and edge devices. Think millions of smartphones training a keyboard prediction model:
- Massive scale: Millions to billions of clients
- Unreliable participants: Devices go offline, have limited battery, unstable connections
- Extreme heterogeneity: Different device capabilities, network speeds, data distributions (non-IID data)
- High privacy stakes: Personal user data never leaves the device
- Small per-client datasets: Each phone has relatively little data
- Asynchronous participation: Only a tiny fraction participate in each round (client selection)
Cross-Silo Federated Learning
This is the institutional scenario where hospitals, banks, or companies collaborate:
- Small scale: Typically 2-100 organizations
- Reliable participants: Stable servers with predictable uptime
- Moderate heterogeneity: Different data distributions but consistent infrastructure
- Institutional privacy: Legal/competitive reasons prevent data sharing (GDPR, HIPAA, business secrets)
- Large per-client datasets: Each hospital or bank has substantial data
- Synchronous participation: All or most silos participate in each round
The cross-device setting demands aggressive communication efficiency and robust handling of dropouts, while cross-silo focuses on strong privacy guarantees (often with secure aggregation and differential privacy) and coordination between fewer, more reliable partners.
Key Takeaway: Cross-device federated learning handles millions of unreliable mobile clients with small datasets, while cross-silo coordinates tens of institutional partners with large, private data repositories—each requiring different optimization and privacy strategies.