Multi-Tenant Async Services

In this lesson, you’ll learn how to implement resource quotas and isolation levels to prevent a single tenant from starving your shared asynchronous executor.

In a multi-tenant Rust service, multiple customers share the same underlying event loop and thread pool. Without proper boundaries, a "noisy neighbor"—a tenant performing heavy computation or overwhelming the I/O bus—can introduce latency spikes for everyone else. Achieving isolation requires moving beyond simple task spawning.

Implementing Resource Quotas

The most effective way to enforce quotas in an async executor is to wrap your task execution in a "weighted semaphore" or a custom Semaphore-based bulkhead. Instead of simply spawning a task onto the runtime, you gate access to that task using an async semaphore assigned to the specific tenant:

use tokio::sync::Semaphore;
use std::sync::Arc;

struct Tenant {
    id: String,
    semaphore: Arc<Semaphore>,
}

async fn run_tenant_task(tenant: Arc<Tenant>, task: impl Future<Output = ()>) {
    // Acquire a "permit" based on the tenant's pre-allocated quota
    let _permit = tenant.semaphore.acquire().await.unwrap();
    task.await;
}

Isolation Levels

Isolation exists on a spectrum. At the Logical Isolation level, you use permit-based throttling (shown above) to ensure no single tenant occupies more than their fair share of total concurrency. At the Execution Isolation level, you move high-priority or resource-intensive tasks to a dedicated tokio::runtime instance.

By partitioning your async runtime, you ensure that even if a tenant’s code triggers a long-running synchronous block or excessive polling, it only saturates its private thread pool, leaving the main system executor untouched. When designing these systems, always prefer "soft" limits (throttling/queuing) over hard kills, as they provide a much smoother user experience during traffic bursts.

Combining these techniques ensures that your service maintains predictable performance for premium tenants, even under heavy load from others.

Key Takeaway: Use asynchronous semaphores to enforce tenant-specific concurrency quotas and partition intensive workloads across dedicated runtimes to achieve true execution isolation.