Multi-Adapter Inference and Switching
What you'll learn: How to serve multiple specialized LoRA adapters from a single base model by hot-swapping lightweight adapter weights.
The Core Idea
Once you've trained multiple LoRA adapters for different tasks (customer support, medical Q&A, creative writing), you don't need separate copies of the base model for each one. Instead, you can load just one base model and swap out the tiny LoRA adapter weights on demand, serving different specialized behaviors without reloading billions of parameters.
Think of it like a Swiss Army knife: one handle (base model) with interchangeable tools (adapters). Each tool weighs almost nothing compared to the handle, so you can carry many tools and snap them in instantly.
How Switching Works
The base model's frozen weights stay in GPU memory. When a new request arrives tagged for a specific task (e.g., "use the medical adapter"), you:
- Unload the current adapter matrices (A and B) from the target modules
- Load the new adapter weights from storage
- Continue inference with the same base model, now behaving according to the new adapter
Since LoRA adapters are typically <1% the size of the base model, loading and switching takes milliseconds rather than seconds. You can even cache multiple adapters in memory simultaneously if space permits, making switching nearly instantaneous.
Multi-Tenant Serving
This enables multi-tenant deployment: one GPU serving dozens of customers, each with their own fine-tuned adapter. The infrastructure becomes dramatically more cost-efficient because you're not duplicating the enormous base model for each specialization.
Key Takeaway: Multi-adapter inference lets you serve many specialized models from one base model by swapping tiny LoRA weights on the fly, enabling efficient multi-tenant LLM deployments with minimal memory overhead.