slime

// Overview

slime is an open-source reinforcement-learning post-training framework from THUDM, built for RL scaling. It provides two capabilities that are meant to reinforce each other: high-performance training, by connecting Megatron with SGLang, and flexible data generation, through custom data-generation interfaces and server-based engines. Megatron training, SGLang rollout, custom data generation, reward computation, verifier feedback and environment interaction all flow through the same training / rollout / Data Buffer path rather than a stack of disconnected services.

Its main claim is that it has been used for real releases rather than only examples: the project states slime is the RL framework behind GLM-5.2, GLM-5.1, GLM-5, GLM-4.7, GLM-4.6 and GLM-4.5, and that it also supports the Qwen series (Qwen3.6, Qwen3.5, Qwen3Next, Qwen3MoE, Qwen3, Qwen2.5), the DeepSeek V3 series (V3, V3.1, R1) and Llama 3. It is deliberately opinionated: by choosing SGLang as the single rollout backend, it can use SGLang-specific serving, routing, caching, disaggregation and weight-sync behaviour directly instead of abstracting over several inference engines.

Both engines are exposed by pass-through rather than wrapper code. Every argument the installed SGLang supports can be used by adding a `--sglang-` prefix, and Megatron arguments are read directly, so parallelism, optimizer and checkpointing options stay available as those projects evolve. Around that, slime adds RL-specific machinery: prefill/decode disaggregation for multi-turn and agentic workloads, delta weight sync for training/inference disaggregation, external rollout engines for setups where serving is managed outside the training job, and session-affinity router policies for multi-turn agents.

The project treats correctness as infrastructure work, maintaining CPU unit tests, contract tests for customization hooks, and GPU end-to-end tests covering dense and MoE models, checkpointing, numerical precision, async rollout and PPO-style workflows, plus documented reproducibility, fault tolerance, tracing and profiling. Several independent systems build on it as an RL substrate, including Dressage (agentic RL by Alibaba Accio), Miles (RadixArk) and vime (maintained by the vLLM project).

// What it does

• Single dataflow for RL post-training: Megatron training and SGLang rollout share one training / rollout / Data Buffer path
• Native argument pass-through — SGLang flags via a `--sglang-` prefix, Megatron flags read directly, so no wrapper layer hides upstream options
• Custom data-generation and rollout interfaces (`--rollout-function-path`, `--custom-generate-function-path`) for search, multi-agent, fully-async and coding-agent RL
• Prefill/decode disaggregation, delta weight sync and external rollout engines for large-model and agentic workloads
• Validated on frontier post-training runs — the project cites the GLM-4.5 through GLM-5.2 releases — with support for Qwen, DeepSeek V3 and Llama 3 model families
• Engineering guardrails as first-class features: CPU unit tests, GPU end-to-end CI, reproducibility, fault tolerance, trace viewer and profiling docs

// Getting started

The project recommends its Docker image, which ships the patched SGLang and Megatron dependencies. The quick-start walkthrough trains GLM-Z1-9B on the dapo-math-17k dataset; NVIDIA H-series GPUs have CI coverage, and B200 uses identical steps.

docker pull slimerl/slime:latest

docker run --rm --gpus all --ipc=host --shm-size=16g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -it slimerl/slime:latest /bin/bash

cd /root/slime
git pull
pip install -e . --no-deps

hf download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414

hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k

hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024

cd /root/slime
source scripts/models/glm4-9B.sh

PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/GLM-Z1-9B-0414 \
    --save /root/GLM-Z1-9B-0414_torch_dist

cd /root/slime
bash scripts/run-glm4-9B.sh

PYTHONPATH=/root/Megatron-LM python tools/convert_torch_dist_to_hf.py \
  --input-dir /path/to/torch_dist_ckpt/iter_xxx/ \
  --output-dir /root/GLM-Z1-9B-0414-iter_xxx \
  --origin-hf-dir /root/GLM-Z1-9B-0414

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Reach for it when you are running RL post-training at release scale on a Megatron + SGLang stack and want the rollout and training loop in one dataflow
• Reach for it when your reward signal comes from tools, sandboxes, verifiers or environments and needs a custom generation function rather than a fixed trainer
• Reach for it for agentic RL — multi-turn, fully-async or coding-agent workloads where prefill and decode have different resource profiles
• Reach for it when you need upstream SGLang or Megatron features immediately, without waiting for a framework abstraction to expose them

// How slime compares

slime alongside other open-source rlhf & alignment tools AI/TLDR tracks, ranked by GitHub stars.