Tencent Opens Hy4 flash, a 12B-Active Model With a 64B N-Gram Memory, Under a License That Makes Big API Hosts Ask First

Tencent has published open weights for Hy4 flash, its cheapest Hy model, on Hugging Face, ModelScope and GitCode, shortly after its November 16, 2026 API launch. The 220B-parameter mixture-of-experts model activates about 12B parameters per token and supports 1M tokens of context. Unlike the Apache-2.0 Hy4 preview, it ships under a custom Tencent Hy License with extra terms for large Model-as-a-Service operators.

A towering honeycomb wall where only a dozen cells glow, while a gatekeeper at an open gate pauses a large freight wagon as small carts pass.
A towering honeycomb wall where only a dozen cells glow, while a gatekeeper at an open gate pauses a large freight wagon as small carts pass.

Tencent’s cheapest Hy model is now downloadable, but not under the license its predecessor used. The Tencent Hy team has published Hy4 flash, an FP8 build and the pretrained Hy4 flash-base as open-weight models on Hugging Face, ModelScope and GitCode, with code on GitHub. The Hugging Face repositories were created on November 17 and were still being updated on November 18 (UTC). Tencent launched Hy4 flash as an API product on November 16, 2026, and said at the time that it planned to open-source the model.

Small active footprint, large memory table

The model card describes Hy4 flash as having 220B total parameters, about 64B of them in “Engram” n-gram embeddings, with roughly 12B active per token. It breaks this down as 155.5B core parameters, 63.5B Engram parameters, a 0.43B vision encoder and a 0.85B built-in drafter; active parameters are 11.3B for text plus 0.43B for vision. Tencent’s November 16 announcement gave the total as 224B. The model takes text and images, generates text, and supports a context window of 1,048,576 tokens.

Its 42-layer backbone has one dense layer and 41 mixture-of-experts layers. Tencent lists three changes from the larger Hy4:

  • Hybrid attention. Sparse attention layers that pick up to 2,048 tokens alternate with 512-token sliding-window layers at a 1:3 ratio.
  • LatentMoE. Routed experts operate in a 1,536-dimension latent space; each token activates 16 of 512 routed experts plus one shared expert.
  • Engram. Learned bigram and trigram embeddings, retrieved by multi-head hash lookups, are injected at four points in the backbone.

A five-layer drafter called DFly2 handles speculative decoding by reusing the main model’s KV cache and the sparse layers’ token selections, which Tencent says keeps drafting cost constant as context grows and adds no extra KV memory. Tencent says the design cuts the KV cache to 1/24 of Hy3 and 1/6 of Hy4, that reinforcement learning halved average output tokens, and that megakernels, 4-bit quantization and DFly2 push decoding past 150 tokens per second.

Cost-per-task claims rest on Tencent’s own tests

Tencent built an internal benchmark, WorkIndex, from typical tasks in its WorkBuddy assistant, including slide creation, document editing, spreadsheet repair and audio and video processing. Hy4 flash scores 42.1, against 39.7 for Hy4 preview and 51.4 for Hy4. Tencent says its average cost per task is 1/13 of Hy4 preview, 1/20 of Hy4 and 1/8 of DeepSeek-V4.1-Flash-1017 at Max tier peak-hour pricing.

In a blind test with 85 internal WorkBuddy users across 259 tasks, Hy4 flash scored 2.79 out of 4 for satisfaction against 2.81 for Claude Haiku 5.5, with a median completion time of 6.7 minutes and a cost of RMB 0.26 per task. Public-benchmark scores appear only in charts.

For the base model, Tencent’s few-shot table shows 88.7 on MMLU, 71.1 on MMLU-Pro, 57.8 on SuperGPQA and 69.8 on U-MATH. That is ahead of DeepSeek-V4-flash-Base on MMLU-Pro, SuperGPQA and U-MATH but a point behind it on MMLU, and below Hy4 Base on every knowledge benchmark listed. The card warns that heavy synthetic instruction data in mid-training sometimes breaks answer formatting in few-shot tests.

On Tencent Cloud’s TokenHub, Hy4 flash lists at about $0.070 per million input tokens, $0.281 per million output tokens and $0.003 per million cached input tokens. Tencent is offering 50% off through November 30 and free use in WorkBuddy over the same period. It recommends the model for deep search, document processing, slides, simple front-end demos and computer use, and points users to Hy4 for long-horizon software engineering.

vLLM and SGLang builds, and a $50 million threshold for API hosts

The repositories are not gated. Tencent recommends vLLM or SGLang, with prebuilt container images; its vLLM example serves the FP8 build across eight GPUs with tensor parallelism. Transformers support has not reached the main branch, so the card installs from a pull request. A GGUF build is offered “for end-to-end testing purposes,” and a fine-tuning guide is included.

The bigger change is legal. Hy4 preview, whose weights went up on Hugging Face in August, carries an Apache-2.0 tag on Hugging Face. Hy4 flash ships under the Tencent Hy License 1.0, which says it is derived from Apache 2.0 but “is not the Apache License.” Its added Section 10 targets Model-as-a-Service businesses, defined as giving third parties remote access to model inference or fine-tuning, for example via an API.

A MaaS operator must obtain a separate written license from Tencent if its worldwide gross revenue exceeds US$50 million in any 12-month period, whether or not that revenue comes from the model, or before using the model in a product with more than 100 million monthly active users. Purely internal use is exempt as long as neither the model nor its outputs reach third parties. End-user products that confine the model to specific features, without general-purpose access, fall outside the MaaS definition.

Topics
  • #Tencent
  • #Hy4 flash
  • #Open Weights
  • #Mixture of Experts
  • #Model License
Raj M

Author

Raj M

Contributor

AI Systems Architect is a seasoned technology leader with over 15 years of experience in the IT industry working with Fortune 500 companies. With a solid foundation in multi-agent systems, open-source LLM infrastructure, and enterprise deployment, he excels at building scalable production-grade AI platforms.