ERNIE 5.5 Flash Base

// Overview

ERNIE 5.5 Flash Base is an open-weight language model from Baidu, published on Hugging Face and ModelScope on November 5, 2026. It is a sparse Mixture-of-Experts model with 280B total parameters and 14B activated per token, pre-trained from scratch on 33.5 trillion tokens with a training context of up to 128K; fine-tuning and inference support a 1M-token context. It is a text-only model released in FP8 and BF16 safetensors.

The architecture has 43 layers: one dense FFN layer and 42 Multi-Router Latent MoE layers, each with 512 routed experts (top-10) plus one shared expert. Attention is hybrid: 37 layers use Heavily Compressed Attention (HCA), which caches one entry per 128 tokens, and 6 layers use Multi-head Latent Attention with sparse attention (MLA + DSA). Baidu also describes Virtual Head Attention, an ECHO layer that reuses the backbone's final layer for multi-token prediction and as a speculative-decoding draft, manifold-constrained Hyper-Connections in place of the standard residual stream, a MultiMax output head and a ≈200K-entry BPE tokenizer that groups digits right-to-left. The model was trained with the Muon optimizer.

This is the checkpoint at the end of pre-training, before any mid- or post-training: it is not an instruction-tuned chat model and is served through the completions endpoint. Baidu positions it as a starting point for domain adaptation, instruction tuning and reinforcement-learning research, and says its pre-training data emphasizes repository-scale code and long-context understanding.

In Baidu's model card, evaluated with one pipeline against DeepSeek-V4-Flash-Base (284B-A13B), Hy3-preview-Base (295B-A21B) and MiMo-V2.5-Base (310B-A15B), it scores 93.3 on MMLU, 78.4 on MMLU-Pro, 60.2 on GPQA, 82.3 on HumanEval+ and 91.7 on RepoQA. It does not lead every row: DeepSeek-V4-Flash-Base is ahead on ChineseSimpleQA, INCLUDE-lite, HELMET-Rerank and InfiniteBench, and Hy3-preview-Base on BBH (CoT) and FullStackBench.

The weights are released under the ERNIE 5.5 Flash Base Model License Agreement v1.0. Non-commercial use is free; commercial use is free of model license charges for operators whose related annual revenue is at or below US$50 million, provided they display "Powered by ERNIE" and send Baidu a one-time use notice. Above that threshold, commercial use needs Baidu's separate written authorization.

// Benchmarks

This model's scores

1. PIQA96%
2. MMLU93.3%
3. RepoQA91.7%
4. CMMLU90%
5. HumanEval+82.3%
6. MMLU-Pro78.4%
7. GPQA60.2%
8. LongBench-v248.7%
9. LiveCodeBench40.1%

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Strengths

• 280B-total / 14B-active MoE with a 1M-token context for fine-tuning and inference
• Released as a raw pre-trained base in FP8 and BF16, for custom instruction tuning and RL
• 93.3 MMLU, 78.4 MMLU-Pro and 82.3 HumanEval+ in Baidu's base-model comparison
• Day-one serving recipes for SGLang and vLLM (via the vLLM-ERNIE plugin)

// Best for

• Reach for it as a starting checkpoint for domain adaptation, instruction tuning or reinforcement-learning research on a large open MoE.
• Reach for it for long-context and repository-scale code work when you plan to post-train the model yourself.
• Look elsewhere for a ready-to-use chat assistant or image input: this is a text-only base model with no instruction tuning.