Soup · 2026-09-04 · notable
Soup v0.74.0 — a dtype bug was doubling every fine-tune's memory
Soup v0.74.0 fixes a bug that loaded the frozen base model in fp32, twice its checkpoint precision. On an H100 running Llama-3.1-8B with LoRA, peak memory falls from 48,241 MiB to 18,658 MiB — 2.59x less.
A missing dtype argument meant Soup loaded the frozen base model at twice its checkpoint precision, on every single fine-tune.
What is it?
Soup v0.74.0 fixes a memory bug that sat in all three `from_pretrained` call sites — text, vision and audio. None of them passed an explicit dtype, so the frozen base model, which never receives an optimizer step, was materialized in fp32. Soup is an open-source CLI that runs LLM fine-tuning and post-training from a single YAML file.
How does it work?
The fix passes the checkpoint's own dtype instead of letting the loader default to fp32. Alongside it, the two independent copies of the full-fine-tune check — one in the trainer, one in the VRAM pre-flight — that used to disagree in both directions are merged into a single shared `is_full_finetune()` helper. A base you actually train still loads in fp32, deliberately.
Why does it matter?
Measured on an H100 with Llama-3.1-8B and LoRA, peak memory drops from 48,241 MiB to 18,658 MiB — 28.9 GB back, byte-identical across three repeats. The same release unblocks the free Colab and Kaggle tier, where layer streaming used to crash on T4, P100, V100 and GTX 16xx cards, and adds a Lambda Cloud training backend.
Who is it for?
people fine-tuning open models on their own GPUs
Try it
pip install -U "soup-cli[train]"