Traffic Splitting and Assignment Logic
What You'll Learn
You'll learn how to consistently route users to the right model variant during A/B tests without creating confusing experiences or biased results.
The Core Challenge
When running A/B tests with multiple model versions, you need two critical guarantees:
- Consistency: Each user always sees the same model variant across sessions
- Fair distribution: Traffic splits evenly (or according to your specified ratios)
Think of this like a restaurant testing two menus. You wouldn't want a customer getting Menu A on Monday, Menu B on Wednesday, then Menu A again on Friday—that's confusing and makes it impossible to measure which menu they prefer.
Assignment Logic
The standard approach uses deterministic hashing:
import hashlib
def assign_user_to_variant(user_id: str, variants: list, split_ratios: list):
"""
Consistently assign user to a variant based on their ID.
split_ratios: [0.5, 0.5] means 50/50 split between two variants
"""
# Hash user_id to get consistent random-looking number
hash_value = int(hashlib.md5(user_id.encode()).hexdigest(), 16)
# Convert to percentage (0-100)
bucket = (hash_value % 100) / 100.0
# Assign based on cumulative thresholds
cumulative = 0
for variant, ratio in zip(variants, split_ratios):
cumulative += ratio
if bucket < cumulative:
return variant
This ensures user_123 always lands in the same bucket, whether they make 1 request or 1,000.
Routing Requests
Once assigned, route to the appropriate model endpoint:
variant = assign_user_to_variant(user_id, ["baseline", "fine_tuned"], [0.8, 0.2])
if variant == "baseline":
response = baseline_model.generate(prompt)
else:
response = fine_tuned_model.generate(prompt)
Key Takeaway: Use deterministic hash-based assignment to ensure users consistently see the same model variant, enabling meaningful A/B test comparisons without user confusion or statistical bias.