# LLaMA-Factory

> Fine-tune 100+ open LLMs and multimodal models with one CLI or a web UI

- Category: Training & Fine-Tuning / Fine-Tuning Frameworks
- Language: Python
- Licence: Apache-2.0
- Repository: https://github.com/hiyouga/LlamaFactory
- Website: https://llamafactory.readthedocs.io
- Canonical page: https://ai-tldr.dev/tools/llama-factory/

## In short

LLaMA-Factory is an open-source training suite that lets you pre-train, supervised fine-tune, and run RLHF on large language models from one place. It sits in the Fine-Tuning Frameworks category of the Training & Fine-Tuning tools and is open-source, written in Python and licensed Apache-2.0. To get started: install from source.

## Overview

LLaMA-Factory is an open-source training suite that lets you pre-train, supervised fine-tune, and run RLHF on large language models from one place. It supports 100+ text and multimodal model families, including LLaMA, Qwen3, Mistral, Mixtral-MoE, DeepSeek, Gemma, GLM, Phi, and LLaVA, and you drive everything through a command-line tool or a Gradio web UI called LLaMA Board.

It is aimed at ML engineers and researchers who want to adapt an existing open model to their own data without writing a custom training loop. You describe a training run in a YAML config and pass it to the CLI, or fill in the fields in the web UI, so most common fine-tuning recipes work with little or no code.

As a fine-tuning framework, it bundles the methods you would otherwise wire together yourself: full and freeze tuning, LoRA and 2-8 bit QLoRA, reward modeling, PPO, DPO, KTO, and ORPO, plus speed-ups like FlashAttention-2, Unsloth, and Liger Kernel. After training it can serve the model through an OpenAI-style API backed by a vLLM or SGLang worker.

## What it does

- Supports 100+ models, including LLaMA, Qwen3, Qwen3-VL, Mistral, Mixtral-MoE, DeepSeek, Gemma, GLM, Phi, and LLaVA
- Covers (continuous) pre-training, (multimodal) supervised fine-tuning, reward modeling, PPO, DPO, KTO, and ORPO
- Scales from 16-bit full tuning and freeze-tuning to LoRA and 2/3/4/5/6/8-bit QLoRA via AQLM, AWQ, GPTQ, and others
- Includes training speed-ups such as FlashAttention-2, Unsloth, Liger Kernel, RoPE scaling, and NEFTune
- Zero-code workflows via the llamafactory-cli command line and the LLaMA Board web UI
- Serves trained models through an OpenAI-style API with a vLLM or SGLang worker, plus experiment tracking via TensorBoard, W&B, MLflow, and SwanLab

## Getting started

Install LLaMA-Factory from source, then run a LoRA fine-tuning, chat, and export cycle using the bundled example configs.

1. Install from source

   Clone the repository and install it in editable mode along with the metrics requirements.

```bash
git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory
pip install -e .
pip install -r requirements/metrics.txt
```

2. Run LoRA fine-tuning

   Train a LoRA adapter using a provided example YAML config. Each config sets the model, dataset, and training arguments.

```bash
llamafactory-cli train examples/train_lora/qwen3_lora_sft.yaml
```

3. Chat with the fine-tuned model and export it

   Load the adapter to chat in the terminal, then merge it into a standalone model.

```bash
llamafactory-cli chat examples/inference/qwen3_lora_sft.yaml
llamafactory-cli export examples/merge_lora/qwen3_lora_sft.yaml
```

4. Use the web UI instead

   Prefer a no-code workflow? Launch LLaMA Board to configure and run training in the browser.

```bash
llamafactory-cli webui
```

## When to use it

- Adapt an open base model like Qwen3 or LLaMA to a domain dataset with LoRA or QLoRA on limited GPU memory
- Align a model to human preferences using DPO, PPO, KTO, or ORPO without building a custom RLHF pipeline
- Fine-tune multimodal models for image understanding, visual grounding, or video and audio tasks
- Let non-experts run and compare training jobs through the LLaMA Board web UI, then serve results via an OpenAI-style API