Horizontal Sharding Strategies
You'll learn how to split your Prometheus scraping workload across multiple servers using functional and hashmod sharding to handle large-scale metrics collection.
The Core Problem
When your infrastructure grows beyond what a single Prometheus server can handle, you need a way to divide the scraping load. Think of it like a restaurant that gets too busy for one chef—you need multiple kitchens (Prometheus instances), each cooking specific dishes (scraping specific targets).
Functional Sharding
Functional sharding divides targets by their function or type. Each Prometheus server scrapes a specific category of services:
- Prometheus-A scrapes all Kubernetes pods
- Prometheus-B scrapes all databases
- Prometheus-C scrapes all application servers
This is straightforward to understand and maintain. You configure each Prometheus instance with different scrape_configs that target specific job labels or service types.
Hashmod Sharding
Hashmod sharding uses a mathematical hash function to distribute targets evenly across Prometheus instances. Each server scrapes only the targets whose hash value matches its assigned shard number.
You configure this using relabel_configs with hashmod to calculate a hash of target labels (like instance address), then keep only targets matching that server's shard ID. For example, with 4 shards, Prometheus-0 keeps targets where hash % 4 = 0, Prometheus-1 keeps hash % 4 = 1, and so on.
This provides better load balancing than functional sharding, especially when target types have uneven resource demands.
Trade-offs
Functional sharding is simpler but can create imbalanced load. Hashmod sharding distributes load more evenly but requires careful coordination when adding or removing shards (targets get reassigned).
Key Takeaway: Horizontal sharding splits Prometheus scraping across multiple servers using either functional sharding (by target type) or hashmod sharding (by hash distribution) to overcome single-instance scalability limits.