Hy4

Tencent released Hy4 preview on August 28, 2026 as an open-weight MoE model from the Tencent Hy Team, according to this third-party analysis, which cites the company's release announcement. The model carries 770 billion total parameters with 49 billion activated per token, a context window of roughly one million tokens, and a built-in multi-token prediction layer used for speculative decoding. Weights are published under the Apache License 2.0 in both full-precision and FP8-quantized checkpoints across Hugging Face, ModelScope, GitCode, and CNB. The analysis flags an important caveat about how Hy4 preview's standing should be read: Tencent's blind comparative evaluation was run internally, so the published benchmark numbers reflect a self-selected internal test suite rather than a settled, third-party-audited ranking. The authors argue these limits do not invalidate the published results, but they narrow how confidently the model can be compared with other open-weight systems, and any capability ranking claims should be treated as vendor self-reported.

Tencent's Hy Team published the official model card for Hy4-preview, a new-generation Mixture-of-Experts flagship model with 770B total parameters and 49B activated per token. The backbone spans 78 layers, with the first using a standard dense FFN and the remaining 77 using MoE blocks containing 256 routed experts plus one shared expert, activating the top-8 routed experts per token. The card states the model was scaled on model size, context length, and training data, with post-training adding to the gain. Architecturally, Hy4-preview uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, four residual streams, and iHC identity Hyper-Connections to widen inter-layer information flow. It includes one native multi-token prediction layer of 10B parameters with 0.7B activated for speculative decoding, a 1M context length, a 120832 vocabulary size, and a 6144 hidden size across 64 attention heads. The card also documents deployment paths including vLLM and SGLang, plus finetuning, quantization, license, and known-limitations sections.