How to Merge a LoRA Adapter Back Into the Base Model

// In plain English

When you fine-tune a model with LoRA, you don't change the original model at all. You train a tiny set of extra weights — the adapter — that sits beside the frozen base model and nudges its behavior. A finished LoRA adapter is often just a few megabytes, while the base model it rides on can be tens of gigabytes.

Once training is done, you face a simple fork in the road. Do you keep the adapter as a separate file that loads on top of the base model at runtime, or do you merge it permanently into the base weights to produce one standalone model? Merging means adding the adapter's learned changes back into the original numbers, so the result behaves exactly like the fine-tuned model but is a single, ordinary checkpoint again — no adapter file required.

Think of the base model as a printed book and the LoRA adapter as a stack of sticky notes with your corrections. Keeping them separate is like handing someone the book plus the notes: flexible, because you can peel the notes off or swap in a different set, but the reader has to hold both. Merging is like printing a brand-new edition with every correction baked into the text: cleaner to hand out, but now the corrections are locked in and you can't easily pull them back off.

// Why it matters

The choice between keep separate and merge is not cosmetic — it decides how you deploy, how fast inference runs, and how easily you can serve many fine-tunes at once. Picking wrong can cost you latency, memory, or a maintenance headache later.

Keeping the adapter separate is what makes LoRA so cheap to operate at scale. You load the big base model once into GPU memory and attach small adapters on top. A single GPU holding one 13B base model can serve dozens of different fine-tunes — one per customer, per language, per task — by swapping multi-megabyte adapters instead of loading multiple multi-gigabyte models. That is the entire economic argument for adapters in production.

Merging matters when you want the opposite: a single, self-contained model with zero runtime overhead. The adapter adds a little extra math to every forward pass, and a merged model removes it — the fine-tuned weights are now just the model's weights. You also get a checkpoint you can hand to any tool that doesn't understand adapters: a plain GGUF for llama.cpp, an export to ONNX, or an upload to a hub where users expect one ready-to-run model.

• Latency-sensitive single model. Merge it. One model, no adapter layers, fastest possible forward pass.
• Many fine-tunes, one base. Keep separate. Load the base once, hot-swap adapters per request — far less memory than N merged models.
• Export to a format that ignores adapters (GGUF, ONNX, some serving runtimes). Merge first, then convert.
• Still iterating / stacking adapters. Keep separate so you can swap, combine, or retrain without touching the base.

// How it works

LoRA never trains a full weight matrix. For a target layer with weight matrix W, it learns two small matrices, A and B, whose product is the change you want — usually scaled by a constant. The effective fine-tuned weight is simply the original plus that learned change:

W_finetuned = W_base + (alpha / r) * (B @ A)

  W_base   : the frozen original weight matrix (large)
  A, B     : the trained low-rank matrices (small)
  r        : the LoRA rank
  alpha    : the LoRA scaling factor
  B @ A    : the low-rank update, same shape as W_base

When you keep the adapter separate, the runtime computes B @ A and adds its effect during every forward pass — the base weights stay untouched on disk and in memory. When you merge, you compute that update once, add it directly into W_base, and throw the adapter away. The result is a normal weight matrix that already contains the fine-tuning. (See LoRA rank and alpha for what r and alpha actually do.)

The merge in code

In the Hugging Face PEFT library the whole operation is two lines. You load the base model, attach the trained adapter, then call merge and unload to get back a plain model you can save.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    torch_dtype="bfloat16",   # load in full/16-bit precision to merge cleanly
)

# Attach the trained LoRA adapter on top of the base.
model = PeftModel.from_pretrained(base, "./my-lora-adapter")

# Fold B@A into the base weights, then drop the adapter layers.
merged = model.merge_and_unload()

# Now it is an ordinary model — save it like any checkpoint.
merged.save_pretrained("./my-merged-model")
AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B").save_pretrained("./my-merged-model")

// The QLoRA precision gotcha

If you trained with QLoRA, there is a quality trap that catches almost everyone the first time. QLoRA loads the base model in 4-bit to save memory, then trains a normal-precision adapter on top. That's great for training — but it changes what "merge" means.

Your adapter (A and B) was trained against the dequantized values the 4-bit base unpacks to at runtime. If you merge that adapter straight into a model that is still stored in 4-bit, the framework has to round the merged result back down to 4-bit. The adapter's careful small corrections get crushed by that re-quantization, and the merged model can come out noticeably worse than the adapter felt during training — sometimes badly enough to look broken.

The rule generalizes: merge in the highest precision you can afford, and quantize as the very last step. Every rounding stage you do after the merge loses a little of what the adapter learned, so you want as few of them as possible — ideally just one, at the end, when the fine-tuning is already locked in.

// Keep separate vs merge: a decision table

There's no universally right answer — it depends on how many fine-tunes you serve and what runs them. This table covers the cases you'll actually hit.

// Merging and stacking multiple adapters

Because each LoRA update is just B @ A added to the base, you can in principle add several adapters to the same model — a math adapter plus a translation adapter, say. PEFT supports combining adapters with weights so each contributes a chosen amount, and you can merge that blend down into one model.

But stacking is not free magic. Each adapter was trained alone, so adding two together can have them step on each other — one may dominate, or the combination may behave worse than either does by itself. Treat any merged blend as a new model that needs its own evaluation, not as a guaranteed sum of skills.

• Serve separately, route per request — keep adapters apart and pick one based on the task. Cleanest when behaviors shouldn't mix.
• Combine then merge — blend adapters with weights into one model when you genuinely want both behaviors at once, then evaluate the result.
• Re-train on combined data — often the most reliable path: instead of merging two adapters, fine-tune one new adapter on data covering both skills.

// Going deeper

Once the basic merge is comfortable, a few subtler points separate a clean deployment from a flaky one.

Always evaluate the merged model, not just the adapter. Numbers can drift slightly during the merge — most often from precision changes, occasionally from a config mismatch. Run your eval set against the merged checkpoint before shipping it. "The adapter looked good in training" is not the same as "the deployed model is good."

Match dtype and config across the round trip. Load the base for merging in the same architecture and a precision at least as high as you'll deploy at. Mismatched torch_dtype, a different base revision, or a wrong target-module list can silently produce a model that loads fine but answers worse. Pin the exact base model version you trained against.

Dynamic adapter serving is now a first-class feature. Modern serving stacks (such as vLLM's multi-LoRA support) load one base model and hot-swap many adapters per request at near-merged speed. This narrows the old latency gap and makes "keep separate" attractive even when you care about throughput — you get the memory savings and fast inference, so don't assume merging is automatically faster in a real serving setup.

Merging is a tool, not a finish line. It's reversible only in the sense that you kept the original adapter file — so archive your adapters even after you merge. From here, the natural next steps are deciding when to fine-tune at all, comparing full fine-tuning vs PEFT, and getting the rank and alpha right before you train, since no merge can fix an under-trained adapter.