Course contentsShow
← Machine Learning and Deep Learning
Lesson 1719 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

Inference with LoRA: Merging Adapters

How to merge LoRA weights into base model for zero-overhead inference after training.

Inference with LoRA: Merging Adapters

What you'll learn: How to combine LoRA adapter weights with the base model to achieve zero-overhead inference after training.

The Efficiency Opportunity

After fine-tuning with LoRA, you have two sets of weights: the frozen base model weights W and your trained low-rank adapter matrices A and B (where the update is BA). During training, you kept these separate to save memory. But for inference, there's a clever optimization available.

The Merging Process

Since LoRA updates are additive, you can merge them directly into the base weights:

W_merged = W + (α/r) × BA

This is simple matrix addition. The adapter matrices B (of rank r × d) and A (of d × r) multiply to form a full-rank update matrix, which you scale by α/r and add to the original weights.

Why This Matters

Once merged, your model is identical to a fully fine-tuned model at inference time:

  • Zero latency overhead: No extra computation layers
  • Same memory footprint: Just the base model size
  • Standard deployment: Use any inference framework

Think of it like writing notes in pencil (LoRA during training—easy to erase and modify) then permanently copying them into the book (merging for deployment).

Adapter Swapping

The beauty: you can keep multiple LoRA adapters unmerged for different tasks, merge whichever one you need on-the-fly, or even switch between them without reloading the entire base model.

Key Takeaway: Merging LoRA adapters into base weights eliminates inference overhead entirely, giving you the benefits of full fine-tuning performance with the efficiency of parameter-efficient training.