Hy4 preview is Tencent's open-weight Mixture-of-Experts large language model, designed as a general-purpose workhorse for real productivity tasks such as coding and office workloads. The architecture pairs 770B total parameters with 49B active per token, organized as a 78-layer backbone where the first layer uses a dense feed-forward network and the remaining 77 layers each carry 256 routed experts plus a shared expert, activating the top 8 routed experts and the shared one at inference time. The attention pathway uses Gated DeepSeek Sparse Attention with an IndexCache for cross-layer sparse index reuse, while the residual stream uses iHC (identity Hyper-Connections), giving the model a long-context window exceeding one million tokens for working with very large documents or codebases.
Tencent positioned Hy4 preview as reaching the top tier of open-source models, combining stronger pre-training scale with an enlarged post-training run. In Tencent's internal blind evaluation with 163 experts across 203 engineering tasks, the model placed first overall among open-source contenders, and an external benchmark tracker places it in the top tier of open models for reasoning, tool calling, coding, and math. It also includes a native Multi-Token Prediction layer that supports speculative decoding for faster inference. The practical fit is a thinking and long-context model aimed at engineers and knowledge workers who need open weights, deep tool use, and very large context rather than lightweight chat.
