Overview
DeepSpec is a full-stack codebase from DeepSeek for training and evaluating draft models for speculative decoding — the technique where a small, fast model proposes tokens that a larger target model verifies, so the expensive model runs fewer forward passes per accepted token. Rather than shipping one drafter, it packages the whole pipeline: data-preparation utilities, draft-model implementations, training code and evaluation scripts.
The workflow runs in three stages that feed each other. Data preparation downloads and splits prompts, regenerates answers with the target model, and builds a target cache — a step with a serious storage footprint, roughly 38 TB for the default Qwen3-4B setting. Training then fits a draft model against those cached target outputs, launching one worker per visible GPU; the default configs assume a single node with eight GPUs. Evaluation measures speculative-decoding acceptance over a standard benchmark set that includes GSM8K, MATH-500, AIME25, HumanEval, MBPP, LiveCodeBench, MT-Bench, AlpacaEval and Arena-Hard-v2.
Three draft-model algorithms are included — DSpark, DFlash and Eagle3 — each with configs under config/ and released checkpoints on Hugging Face for Qwen3-4B, Qwen3-8B, Qwen3-14B and Gemma-4-12B-it, matching the paper's results table. The repository is MIT-licensed and builds on earlier open work, notably SpecForge for the training framework and Eagle3 implementation and the DFlash draft-model recipe, with attribution recorded in its NOTICE file.
What it does
- End-to-end pipeline: data preparation, draft-model training, and speculative-decoding evaluation in one repository
- Three drafter algorithms included — DSpark, DFlash and Eagle3 — selected by pointing config_path at a config file
- Released draft checkpoints on Hugging Face for Qwen3-4B, Qwen3-8B, Qwen3-14B and Gemma-4-12B-it targets
- Acceptance measured over GSM8K, MATH-500, AIME25, HumanEval, MBPP, LiveCodeBench, MT-Bench, AlpacaEval and Arena-Hard-v2
- Multi-GPU training that spawns one worker per visible device, with per-field config overrides via --opts
- MIT-licensed, with third-party attribution to SpecForge and DFlash recorded in NOTICE
Getting started
Run the three stages in order — each stage's output feeds the next. Note the storage requirement before starting data preparation, and check the script headers for the full config list.
Install dependencies
Data preparation additionally needs an inference engine to serve the target model while answers are regenerated; see scripts/data/README.md.
python -m pip install -r requirements.txtPrepare the data
Download and split the training data, regenerate answers with the target model, and build the target cache. This cache is large — roughly 38 TB for the default Qwen/Qwen3-4B setting.
Train a draft model
train.sh launches train.py, which spawns one worker per visible GPU. Point config_path at one of the configs under config/ to choose the algorithm and target model. Checkpoints land in ~/checkpoints/<project_name>/<exp_name>/step_*.
bash scripts/train/train.shEvaluate acceptance
eval.sh runs eval.py against a trained checkpoint over the benchmark set. Set target_name_or_path to the target the draft was trained against, and draft_name_or_path to your checkpoint or one of the released Hugging Face repo IDs.
bash scripts/eval/eval.shCommands and code are distilled from the project's own documentation — always check the official repo for the latest.
When to use it
- Train a draft model for a target you already serve, so speculative decoding cuts its latency on your own workload
- Compare DSpark, DFlash and Eagle3 under one training and evaluation setup instead of three separate codebases
- Reproduce or extend the published acceptance numbers using the released checkpoints and matching configs
- Fine-tune a drafter for a specific domain, which the authors recommend over reusing a general checkpoint
How DeepSpec compares
DeepSpec alongside other open-source efficient training tools AI/TLDR tracks, ranked by GitHub stars.
| Tool | Stars | What it does |
|---|---|---|
| DeepSpeed | ★ 43.1k | A deep learning optimization library whose ZeRO memory partitioning and offloading let you train very large models across many GPUs. |
| Megatron-LM | ★ 17.8k | NVIDIA's library for training large transformer models at scale using tensor, pipeline, and sequence parallelism. |
| Accelerate | ★ 9.9k | A library that runs the same PyTorch training code across CPUs, multiple GPUs, and TPUs while handling mixed precision, FSDP, and DeepSpeed. |
| DeepSpec | ★ 7.1k | Train and evaluate draft models for speculative decoding |
| TorchTitan | ★ 5.7k | A PyTorch-native platform for pre-training large models that combines FSDP, tensor, pipeline, and context parallelism in one codebase. |
| Nanotron | ★ 2.8k | Hugging Face's minimal library for pre-training LLMs with 3D parallelism, designed to be readable and easy to modify. |