Hy4

Tencent's Hy Team released and open-sourced Hy4 preview on 28 August 2026 as a 770B-parameter MoE model activating 49B parameters per token, with 78 layers, 256 routed experts plus a shared expert, top-8 routing, and a native multi-token prediction layer of 10B total and 0.7B activated parameters for speculative decoding. Context length is 1M tokens with a 120,832-token vocabulary. Weights ship in BF16 as tencent/Hy4-preview and FP8 as tencent/Hy4-preview-FP8 under Apache 2.0, with vLLM and SGLang serving recipes and prebuilt container images. Tencent reports an internal blind evaluation of 163 experts over 203 engineering tasks, scoring 2.99 versus GLM 5.3 at 2.92 and Kimi K3 at 2.94, and lists limitations including over-long reasoning and self over-verification.

Tencent released Hy4 preview, an open-weight MoE language model from the Tencent Hy Team, on August 28, 2026 under the Apache 2.0 license, with 770B total parameters and 49B activated per token. Weights are published as both a full-precision checkpoint and an FP8-quantized checkpoint on Hugging Face, ModelScope, GitCode, and CNB, and the model supports roughly a 1M-token context window. A built-in multi-token prediction layer supports speculative decoding, and the backbone routes each token through a small fraction of total parameters. The analysis notes Tencent's blind comparative evaluation was conducted internally, so benchmark standing should be read as a snapshot of a self-selected internal test suite rather than a settled third-party-audited ranking.

Tencent's Hy Team published the Hy4 preview model card on Hugging Face, describing a new-generation Mixture-of-Experts flagship with 770B total parameters and 49B activated per token. The 78-layer backbone uses a dense FFN for the first layer and 77 MoE layers with 256 routed experts plus one shared expert, activating the top-8 routed experts alongside the shared expert on every token. Attention uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, and the residual path applies identity Hyper-Connections. A native MTP layer of 10B total parameters (0.7B activated) supports speculative decoding, with 1M token context, 6144 hidden size, and 120,832 vocabulary. The card also covers vLLM and SGLang deployment, finetuning, quantization, licensing, and known limitations.