Course contentsShow
← AI Engineering
Lesson 1342 of 1,88632. Fine-Tuning for ProductionFree lesson

Traffic Splitting and Assignment Logic

Implement consistent user assignment to treatment groups and route requests to appropriate model versions.

Traffic Splitting and Assignment Logic

What You'll Learn

You'll learn how to consistently route users to the right model variant during A/B tests without creating confusing experiences or biased results.

The Core Challenge

When running A/B tests with multiple model versions, you need two critical guarantees:

  1. Consistency: Each user always sees the same model variant across sessions
  2. Fair distribution: Traffic splits evenly (or according to your specified ratios)

Think of this like a restaurant testing two menus. You wouldn't want a customer getting Menu A on Monday, Menu B on Wednesday, then Menu A again on Friday—that's confusing and makes it impossible to measure which menu they prefer.

Assignment Logic

The standard approach uses deterministic hashing:

import hashlib

def assign_user_to_variant(user_id: str, variants: list, split_ratios: list):
    """
    Consistently assign user to a variant based on their ID.
    
    split_ratios: [0.5, 0.5] means 50/50 split between two variants
    """
    # Hash user_id to get consistent random-looking number
    hash_value = int(hashlib.md5(user_id.encode()).hexdigest(), 16)
    
    # Convert to percentage (0-100)
    bucket = (hash_value % 100) / 100.0
    
    # Assign based on cumulative thresholds
    cumulative = 0
    for variant, ratio in zip(variants, split_ratios):
        cumulative += ratio
        if bucket < cumulative:
            return variant

This ensures user_123 always lands in the same bucket, whether they make 1 request or 1,000.

Routing Requests

Once assigned, route to the appropriate model endpoint:

variant = assign_user_to_variant(user_id, ["baseline", "fine_tuned"], [0.8, 0.2])

if variant == "baseline":
    response = baseline_model.generate(prompt)
else:
    response = fine_tuned_model.generate(prompt)

Key Takeaway: Use deterministic hash-based assignment to ensure users consistently see the same model variant, enabling meaningful A/B test comparisons without user confusion or statistical bias.