Tencent Hy4

Tencent's Hy Team released Hy4 preview, an open-weight mixture-of-experts language model, on August 28, 2026. The model has 770 billion total parameters with only 49 billion activated per token, and it is distributed under the Apache License 2.0, so developers can modify and redistribute it without restrictive terms. Weights are published on Hugging Face, ModelScope, GitCode, and CNB in both full-precision Hy4-preview and Hy4-preview-FP8 quantized checkpoints. The architecture routes each token through a small fraction of its parameters, supports roughly a one-million-token context window, and includes a built-in multi-token prediction layer for speculative decoding.