Hy4

Tencent released Hy4 Preview, an open-weight text model with 770 billion total parameters, about 49 billion active in a mixture-of-experts design, and a 1 million token context window. The weights occupy 1.56 TB on Hugging Face, roughly five times the size of Tencent's earlier Hy3 model, which had 295 billion parameters and a 256K context. Noting a detail in the chat template file, Simon Willison found the model expects a reasoning effort setting of either high, which is the default, or no think, so chain-of-thought behavior is built in. The model is text-only with no vision support, and its hidden reasoning traces appear in slightly truncated English, which the author attributes to token-efficiency tuning.

Tencent pushed a new open-weight language model to Hugging Face on Friday, August 28, 2026, and developers began downloading it within hours. The Hy4 preview packs 770 billion total parameters into a mixture-of-experts design that activates only about 49 billion of them per request, according to Tencent's own launch post. Tencent said the model targets software engineering, scientific research, and financial analysis, and published a lower-precision Hy4 preview-FP8 variant alongside the full weights for smaller-footprint local or edge use. Weights also appeared on ModelScope, GitCode, and CNB for developers who cannot easily reach Hugging Face from mainland China.

Tencent released Hy4 preview on August 28, 2026 as an open-weight MoE model from the Tencent Hy Team, according to this third-party analysis, which cites the company's release announcement. The model carries 770 billion total parameters with 49 billion activated per token, a context window of roughly one million tokens, and a built-in multi-token prediction layer used for speculative decoding. Weights are published under the Apache License 2.0 in both full-precision and FP8-quantized checkpoints across Hugging Face, ModelScope, GitCode, and CNB. The analysis flags an important caveat about how Hy4 preview's standing should be read: Tencent's blind comparative evaluation was run internally, so the published benchmark numbers reflect a self-selected internal test suite rather than a settled, third-party-audited ranking. The authors argue these limits do not invalidate the published results, but they narrow how confidently the model can be compared with other open-weight systems, and any capability ranking claims should be treated as vendor self-reported.

Tencent's Hy Team published the official model card for Hy4-preview, a new-generation Mixture-of-Experts flagship model with 770B total parameters and 49B activated per token. The backbone spans 78 layers, with the first using a standard dense FFN and the remaining 77 using MoE blocks containing 256 routed experts plus one shared expert, activating the top-8 routed experts per token. The card states the model was scaled on model size, context length, and training data, with post-training adding to the gain. Architecturally, Hy4-preview uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, four residual streams, and iHC identity Hyper-Connections to widen inter-layer information flow. It includes one native multi-token prediction layer of 10B parameters with 0.7B activated for speculative decoding, a 1M context length, a 120832 vocabulary size, and a 6144 hidden size across 64 attention heads. The card also documents deployment paths including vLLM and SGLang, plus finetuning, quantization, license, and known-limitations sections.