Liger-Kernel

// Overview

Liger Kernel is a collection of Triton kernels built specifically for training large language models. It ships Hugging Face-compatible versions of common layers such as RMSNorm, RoPE, SwiGLU, CrossEntropy, and FusedLinearCrossEntropy, fusing them so they run faster and use less GPU memory than the stock PyTorch implementations.

It is aimed at engineers and researchers who fine-tune or pre-train LLMs and are hitting throughput limits or out-of-memory errors. According to the project, it can increase multi-GPU training throughput by about 20% and reduce memory usage by around 60%, which leaves room for longer context lengths, larger batch sizes, and bigger vocabularies.

As an efficient-training tool, it works alongside the stack you already use: Flash Attention, PyTorch FSDP, and Microsoft DeepSpeed. You can adopt it with a one-line patch on top of Hugging Face Transformers rather than rewriting your training loop, and it also includes optimized post-training loss kernels (DPO, ORPO, SimPO, KTO, and more) that cut memory for alignment and distillation work.

// What it does

• Hugging Face-compatible Triton kernels for RMSNorm, RoPE, SwiGLU, CrossEntropy, and FusedLinearCrossEntropy
• One-line adoption via AutoLigerKernelForCausalLM, or per-architecture patch helpers like apply_liger_kernel_to_llama
• Reported ~20% higher multi-GPU training throughput and ~60% lower memory use
• Works with Flash Attention, PyTorch FSDP, and DeepSpeed out of the box
• Post-training loss kernels (DPO, ORPO, CPO, SimPO, KTO, JSD) with up to 80% memory savings
• Runs on NVIDIA (CUDA) and AMD (ROCm) GPUs

// Getting started

Install the package, then enable Liger's kernels with a single change to how you load your model.

pip install liger-kernel

from liger_kernel.transformers import AutoLigerKernelForCausalLM

model = AutoLigerKernelForCausalLM.from_pretrained("path/to/some/model")

import transformers
from liger_kernel.transformers import apply_liger_kernel_to_llama

apply_liger_kernel_to_llama()

model = transformers.AutoModelForCausalLM.from_pretrained("path/to/llama/model")

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Fine-tuning a Hugging Face LLM (e.g. LLaMA 3-8B) that runs out of memory at longer context lengths
• Raising multi-GPU training throughput on FSDP or DeepSpeed without changing your training loop
• Running alignment or preference tuning (DPO, ORPO, SimPO, KTO) with lower memory using the fused post-training loss kernels
• Training with larger batch sizes or bigger vocabularies on the same hardware budget

// How Liger-Kernel compares

Liger-Kernel alongside other open-source efficient training tools AI/TLDR tracks, ranked by GitHub stars.