Hugging Face · 2026-06-18 · notable
Beyond LoRA — Hugging Face benchmark shows OFT and BEFT can beat the default
Hugging Face benchmarked five PEFT methods on identical tasks and found OFT beats LoRA on image fine-tuning while BEFT and Lily trade memory for accuracy on math. PEFT also gained an adapter-to-LoRA converter for vLLM compatibility.
OFT, BEFT, and Lily can beat LoRA — and PEFT now converts them all to LoRA format for vLLM.
Key specs
| Lora adoption | 98.4% of model cards |
|---|---|
| Benchmarks | MetaMathQA + cat-plushy concept learning |
| New peft feature | adapter-to-LoRA converter |
What is it?
'Beyond LoRA' is a Hugging Face benchmark study, published June 18, 2026, comparing parameter-efficient fine-tuning methods head-to-head. The post pairs the comparison with a new PEFT library feature: any supported adapter can be converted to a LoRA on save, which lets downstream packages like vLLM load it without special handling. Authors: Benjamin Bossan, Sayak Paul, Marian Dosmann, and Kashif Rasul.
How does it work?
The Beyond LoRA benchmark runs each method under matched compute and rank on two tasks: MetaMathQA fine-tuning for language models and a cat-plushy concept-learning task for image generation. Beyond LoRA reports accuracy, memory, and training time per method, surfacing tradeoffs that single-method papers rarely show side by side. The PEFT release adds `convert_to_lora()` so non-LoRA adapters can serve through any LoRA-aware runtime.
Why does it matter?
LoRA is on 98.4% of PEFT model cards, but Beyond LoRA shows that default is not always the best — OFT wins on image tasks and BEFT or Lily can win on math when memory is tight. The conversion feature removes the deployment penalty that previously locked teams into LoRA, opening up real method choice for the first time.
Who is it for?
ML practitioners fine-tuning open models, infra teams running vLLM, researchers comparing PEFT methods.
Try it
pip install -U peft