Baidu's 280B ERNIE 5.5 Flash Base Activates Only 14B Per Token and Stops Before Instruction Tuning

Baidu has published ERNIE 5.5 Flash Base, a sparse mixture-of-experts checkpoint that carries 280 billion total parameters and activates only about 14 billion per token, released as the end of pre-training with no mid-training or post-training applied. The company blog and the Hugging Face card for baidu/ERNIE-5.5-Flash-Base, both tied to the November 5, 2026 window, describe a model pretrained from scratch on 33.5 trillion tokens with the Muon optimizer under FP8 mixed precision. Training context reached 128K tokens; fine-tuning and inference are documented to 1 million tokens. Weights ship in FP8 and BF16 on Hugging Face and ModelScope. Serving notes cover SGLang and vLLM through a completions API, not a chat template—consistent with a raw base rather than an instruction-tuned assistant.

That base-only stance is the product pitch as much as the MoE math. Many open releases fold SFT and preference tuning into the same drop. Baidu is shipping the pre-training endpoint itself so labs can run their own mid- and post-training stacks on a fixed sparse backbone. The architecture is 43 layers: one dense FFN layer plus 42 Multi-Router Latent MoE layers, with hidden size 4096 and 64 attention heads. Attention is split across 37 Heavily Compressed Attention (HCA) layers that keep one entry per 128 tokens with 512-dimensional heads, and 6 MLA+DSA layers with 256-dimensional heads. The MoE block routes across 512 experts with top-10 selection plus one shared expert; expert computation lives in a 2048-dimensional latent space, and several sigmoid routers (Multi-Router) decide which experts fire. Additional pieces named in the release include Low-Rank Gated Attention, VHA, mHC with expansion rate 4, a MultiMax output head, and ECHO on the final layer, which predicts future tokens and supplies drafts for speculative decoding. ECHO adds about 0.36B parameters that Baidu excludes from the headline 280B / 14B counts. The tokenizer is a ~201,216-vocab BPE; numbers are encoded right-to-left in groups of up to three digits.

Unified harness scores against three sparse peers

Baidu’s own unified evaluation harness puts Flash Base next to DeepSeek-V4-Flash-Base (284B-A13B), Hy3-preview-Base (295B-A21B), and MiMo-V2.5-Base (310B-A15B). On that board the model posts MMLU 93.3, MMLU-Pro 78.4, GPQA 60.2, HumanEval+ 82.3, MBPP+ 81.5, LiveCodeBench 40.1, LongBench-v2 48.7, and RepoQA 91.7. It trails DeepSeek on a few long-context and Chinese cells Baidu still publishes: HELMET-Rerank 56.2 versus DeepSeek’s 60.0, ChineseSimpleQA 73.0 versus 75.3, and InfiniteBench 46.5 versus 48.0. Those numbers are Baidu’s harness, not an independent third-party board, but the company is explicit about both the wins and the losses on the same table.

License geography and the $50M commercial line

The legal wrapper is the ERNIE 5.5 Flash Base Model License Agreement v1.0, dated November 5, 2026. Territory excludes the European Union, the United Kingdom, and South Korea. Non-commercial use is free. Commercial use is free at or below US$50 million in annual related revenue, with a one-time notice requirement and mandatory “Powered by ERNIE” attribution. Above that revenue line, written authorization is required. Section 5.5 restricts distillation into unrelated models. For builders outside the excluded territories, the practical package is a sparse 280B base with documented 1M-token inference paths, dual-precision weights, and a license that stays free for smaller commercial shops while gating large-revenue and distillation cases.

Serving stays on the completions path in the documented SGLang and vLLM setups, matching a base that has never seen chat templates. Multi-turn assistants need separate SFT; long-context completion and repo-level coding probes can use the sparse stack as-is.

Flash Base is therefore less a ready chat model than a sparse pre-training artifact: 14B active compute per token, a Multi-Router latent MoE stack, ECHO draft support for speculative decoding, and a clear invitation to run custom mid- and post-training on top. Teams that need an instruction-tuned ERNIE will have to wait for Baidu’s follow-ons or train their own; teams that want the pre-training endpoint itself now have FP8 and BF16 checkpoints, SGLang and vLLM serving notes, and a published comparison against three other large sparse bases under one harness.