EAGLE

// Overview

EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) is a speculative-decoding method for making large language models generate text faster, and this repository is its official implementation from SafeAI Lab. Instead of running a separate small model as the drafter, EAGLE attaches a lightweight draft head to the target model that works from the model's own internal feature vectors to guess several tokens ahead; the full model then verifies those guesses in one pass. Because verification decides what is kept, the method is designed to preserve the distribution of the text the original model would have produced — the speed-up comes without a change in output quality.

The repository covers three generations of the idea. EAGLE-1 extrapolates the second-top-layer features of the LLM. EAGLE-2 uses the draft model's confidence scores to approximate acceptance rates and reshapes the draft tree on the fly. EAGLE-3 drops the feature-prediction constraint, simulates it during training with a "training-time test", and fuses low-, mid- and high-level features instead of relying on the top layer alone. The main branch implements EAGLE-3 and EAGLE-2; EAGLE-1 lives on the `v1` branch.

In practice you pair a base model from Hugging Face with a matching EAGLE draft checkpoint. The project publishes official EAGLE-3 weights for Vicuna-13B, Llama 3.1 8B, Llama 3.3 70B and DeepSeek-R1-Distill-Llama-8B, plus EAGLE-1 weights for Vicuna, Llama 2/3 chat, Mixtral and Qwen2 models, and its README lists community checkpoints for Qwen3, Llama 4, GPT-OSS and others. You can then chat with the pair through a Gradio web UI, call an `eagenerate` method that mirrors Hugging Face's `generate`, benchmark it on MT-bench, or train your own draft head.

The method has spread beyond this codebase: the README lists EAGLE support in serving and optimization frameworks including vLLM, SGLang, NVIDIA TensorRT-LLM, TensorRT Model Optimizer, NeMo, MLC-LLM, AWS NeuronX, AMD ROCm and PaddleNLP. For training EAGLE-3 heads to use with SGLang, the maintainers recommend SpecForge. The repository itself is Python and Apache-2.0.

// What it does

• Official implementations of EAGLE-1, EAGLE-2 (dynamic draft trees) and EAGLE-3 (multi-level feature fusion with training-time test)
• Designed to keep the base model's output distribution while generating several tokens per verification step, including non-greedy sampling
• `EaModel.from_pretrained` plus `eagenerate`, a drop-in counterpart to Hugging Face `generate`, with weights spread across multiple GPUs automatically
• A Gradio web UI for chatting with a base model and its EAGLE head, with a tunable number of draft tokens (or `-1` to let EAGLE-2 choose)
• DeepSpeed training code for EAGLE-3 draft heads and MT-bench scripts that measure speed against vanilla autoregressive decoding
• Published draft weights for Vicuna, Llama 2/3/3.1/3.3, Mixtral, Qwen2 and DeepSeek-R1-Distill models, with community checkpoints for many more

// Getting started

The repository runs as a Python package in its own virtual environment. You need a base model and a matching EAGLE checkpoint; the README's tables list which draft weights go with which base model.

git clone https://github.com/SafeAILab/EAGLE.git
cd EAGLE
python -m venv ~/venvs/ea_env
source ~/venvs/ea_env/bin/activate
pip install -r requirements.txt

python -m eagle.application.webui --ea-model-path [path of EAGLE weight] \
  --base-model-path [path of the original model] \
  --model-type [vicuna\llama2\llama3] \
  --total-token [int]

from eagle.model.ea_model import EaModel
from fastchat.model import get_conversation_template

model = EaModel.from_pretrained(
    base_model_path=base_model_path,
    ea_model_path=EAGLE_model_path,
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True,
    device_map="auto",
    total_token=-1
)
model.eval()
conv = get_conversation_template("vicuna")
conv.append_message(conv.roles[0], "Hello")
conv.append_message(conv.roles[1], None)
prompt = conv.get_prompt()
input_ids = model.tokenizer([prompt]).input_ids
input_ids = torch.as_tensor(input_ids).cuda()
output_ids = model.eagenerate(input_ids, temperature=0.5, max_new_tokens=512)
output = model.tokenizer.decode(output_ids[0])

python -m eagle.evaluation.gen_ea_answer_llama3chat --ea-model-path yuhuili/EAGLE3-LLaMA3.1-Instruct-8B --base-model-path meta-llama/Llama-3.1-8B-Instruct --use_eagle3

python -m eagle.evaluation.gen_baseline_answer_llama3chat --ea-model-path yuhuili/EAGLE3-LLaMA3.1-Instruct-8B --base-model-path meta-llama/Llama-3.1-8B-Instruct

cd eagle/traineagle3
deepspeed main.py --deepspeed_config ds_config.json

Commands and code are distilled from the project's own documentation — always check the official repo for the latest.

// When to use it

• Reach for it when generation latency is the bottleneck and you want faster decoding from an existing LLM without accepting a change in its outputs
• Reach for it to train an EAGLE draft head for a model that has no published checkpoint, then deploy it in a serving engine that supports EAGLE
• Reach for it to benchmark speculative decoding on your own hardware and model before enabling it in production
• Reach for it when you are researching speculative decoding and need the reference code behind EAGLE-1, EAGLE-2 and EAGLE-3

// How EAGLE compares

EAGLE alongside other open-source serving & deployment tools AI/TLDR tracks, ranked by GitHub stars.