Hy4

IntuitionLabs reports that Tencent released and open-sourced Hy4 preview on August 28, 2026, describing it as a Mixture-of-Experts model with 770B total parameters and 49B activated per token. The release is under Apache License 2.0, with weights published both as a full-precision Hy4-preview checkpoint and an FP8-quantized Hy4-preview-FP8 checkpoint across Hugging Face, ModelScope, GitCode, and CNB. The analysis notes the model supports roughly a one-million-token context window and includes a built-in multi-token prediction layer for speculative decoding. It also flags that Tencent's blind comparative evaluation was conducted internally, so the model's benchmark standing should be read as a snapshot of a self-selected internal test suite rather than a third-party-audited ranking.

Tencent's Hy Team published Hy4 preview, a new-generation Mixture-of-Experts flagship model with 770B total parameters of which 49B are activated per token. The backbone spans 78 layers: the first uses a standard dense FFN, while the remaining 77 layers replace it with MoE blocks containing 256 routed experts plus one shared expert, activating the top-8 routed experts alongside the shared expert on each token. Beyond the backbone, the model includes one native MTP layer (10B total parameters, 0.7B activated) built in for speculative decoding, and its attention module uses Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse. The residual pathway uses iHC identity Hyper-Connections, the context length is 1M tokens, and the vocabulary size is 120,832.