█

AI/TLDR

LitGPT

Pretrain, finetune, and deploy 20+ LLMs from scratch with no abstractions

github.com/Lightning-AI/litgpt★ 13.4klightning.ai

Overview

LitGPT is an open-source library from Lightning AI for working with large language models. It ships clean, from-scratch implementations of 20+ models such as Llama 3, Phi, Qwen2.5, Gemma, Mistral, and Mixtral, with no hidden abstraction layers so you can read and debug the code directly.

With a single command line tool you can load a model, finetune it on your own data, pretrain from scratch, evaluate it, chat with it, or serve it as an API. It supports memory-saving tricks like LoRA, QLoRA, quantization (fp4/8/16/32), Flash Attention, and FSDP, so the same recipes scale from one GPU up to 1000+ GPUs or TPUs. LitGPT is released under the Apache 2.0 license.

What it does

  • From-scratch, single-file implementations of 20+ LLMs (Llama 3, Phi, Qwen2.5, Gemma, Mistral, Mixtral, and more) with no abstraction layers
  • One CLI for the full model lifecycle: finetune, pretrain, evaluate, chat, and serve
  • Memory-efficient training with LoRA, QLoRA, adapters, and fp4/8/16/32 quantization for low-memory GPUs
  • Scales from a single GPU to 1000+ GPUs or TPUs using Flash Attention and FSDP
  • Tested, ready-to-use training and finetuning recipes configurable through YAML files
  • Simple Python API to load any supported model and generate text in a few lines

Getting started

LitGPT installs from PyPI and gives you both a Python API and a command line tool. The steps below come straight from the project README.

Install LitGPT

Install the package with its extra dependencies using pip.

bashbash
pip install 'litgpt[extra]'

Load a model in Python

Use the Python API to load any supported model and generate text. Weights download automatically.

pythonpython
from litgpt import LLM

llm = LLM.load("microsoft/phi-2")
text = llm.generate("Fix the spelling: Every fall, the family goes to the mountains.")
print(text)

Finetune on your own data

Point the finetune command at a JSON dataset and an output directory. Weights are downloaded automatically.

bashbash
litgpt finetune microsoft/phi-2 \
  --data JSON \
  --data.json_path my_custom_dataset.json \
  --data.val_split_fraction 0.1 \
  --out_dir out/custom-model

Serve the model as an API

Start a web server for a built-in model or your own trained checkpoint so apps can query it.

bashbash
litgpt serve out/custom-model/final

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

When to use it

  • Finetuning an open LLM on your own domain data to build a specialized assistant
  • Pretraining a small language model from scratch on a custom text corpus
  • Deploying a finetuned model as a local inference API for your application
  • Evaluating model quality on benchmarks like MMLU and TruthfulQA before shipping

How LitGPT compares

LitGPT alongside other open-source fine-tuning frameworks tools AI/TLDR tracks, ranked by GitHub stars.

ToolStarsWhat it does
LLaMA-Factory★ 72.3kAn end-to-end training suite with a web UI that covers pre-training, supervised fine-tuning, and RLHF for hundreds of LLMs and multimodal models.
Unsloth★ 66.9kA library that speeds up LoRA and QLoRA fine-tuning while cutting memory use, aimed at training models on a single GPU.
PEFT★ 21.3kHugging Face's library of parameter-efficient fine-tuning methods such as LoRA, DoRA, and prompt tuning that train small adapters instead of full models.
FinGPT★ 20.5kFinGPT is an open-source project of financial LLMs, fine-tuned with LoRA on news and tweet data for tasks like sentiment analysis, relation extraction, and stock-move forecasting.
ms-swift★ 14.6kModelScope's framework for fine-tuning and deploying 600+ LLMs and 300+ multimodal models, supporting PEFT and full-parameter SFT, DPO, and GRPO.
LitGPT★ 13.4kPretrain, finetune, and deploy 20+ LLMs from scratch with no abstractions
Axolotl★ 12.1kA config-driven tool for fine-tuning and post-training open LLMs that supports SFT, LoRA/QLoRA, DPO, GRPO, and multi-GPU training across many model families.
Ludwig★ 11.7kLudwig is a low-code framework that lets you train, fine-tune, and deploy LLMs, multimodal, and tabular models using a YAML config instead of boilerplate Python.