Soup

// Overview

Soup is a free, open-source command-line tool for fine-tuning and post-training large language models. You describe the whole job in one `soup.yaml` file — the base model, the dataset, the task, the LoRA settings — and run `soup train`. Batch size, GPU detection and quantization are chosen for you, and nothing needs an SSH session into a GPU box.

Its distinguishing feature is layer streaming, an opt-in beta mode that keeps the frozen base model in system RAM and feeds it to the GPU one decoder layer at a time. On an RTX 3050 Laptop with 4 GB of VRAM, the project measured Llama-3.1-8B-Instruct with NF4 quantization running at 119.6 tokens per second in a 3.32 GB peak, bit-exact against a normal resident run. Layer streaming also covers the preference losses DPO, ORPO, SimPO and KTO by reusing the same streamed base with its adapters switched off, rather than loading a second reference model.

Within the fine-tuning framework category, Soup aims at the single-machine end: it is a CLI plus a config schema rather than a training library you import. It covers supervised fine-tuning, preference tuning (DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO, BCO), tool-calling, pre-training and distillation, and it can merge adapters and export to GGUF, ONNX, TensorRT, AWQ, GPTQ or BitNet when the run is finished.

// What it does

• One YAML config plus `soup train` — batch size, GPU detection and quantization are handled automatically
• Layer streaming (beta): keeps the frozen base in RAM and streams decoder layers to VRAM, so an 8B model fits a 4 GB card
• Seventeen starting templates, including `chat`, `code`, `tool-calling`, `medical`, `reasoning`, `vision`, `moe` and `longcontext`
• Supervised fine-tuning plus preference tuning with DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO and BCO
• A wide PEFT menu — DoRA, LoRA+, rsLoRA, VeRA, OLoRA, NEFTune, PiSSA, ReLoRA, GaLore — and multi-GPU training via DeepSpeed or FSDP
• Adapter merging and export to GGUF, ONNX, TensorRT, AWQ, GPTQ and BitNet
• `soup reward synth` builds a deterministic, committable reward verifier from a JSONL of reference outputs and refuses to emit one that fails calibration

// Getting started

Soup is a Python 3.10–3.12 command-line application. Install it with its own environment, create a config from a template, then train and test the result. The light install covers config and data commands; fine-tuning needs the `[train]` extra.

pipx install "soup-cli[train]"

# or, inside an existing environment
pip install "soup-cli[train]"

soup init                       # interactive wizard
soup init --template chat       # or start from a template

base: meta-llama/Llama-3.1-8B-Instruct
task: sft

data:
  train: ./data/train.jsonl
  format: alpaca
  val_split: 0.1

training:
  epochs: 3
  lr: 2e-5
  batch_size: auto
  lora:
    r: 64
    alpha: 16
  quantization: 4bit

output: ./output

soup train --config soup.yaml
soup chat  --model ./output
soup push  --model ./output --repo you/my-model

soup merge  --adapter ./output
soup export --model ./output --format gguf --quant q4_k_m

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Fine-tune an 8B open model on a laptop GPU with as little as 4 GB of VRAM using layer streaming
• Adapt a base model to your own chat, code or tool-calling data with a single YAML file and no training loop to write
• Run preference tuning with DPO, ORPO, SimPO or KTO without loading a second reference model
• Export a finished adapter to GGUF for Ollama or llama.cpp, or merge it back into the base model

// How Soup compares

Soup alongside other open-source fine-tuning frameworks tools AI/TLDR tracks, ranked by GitHub stars.