Hunyuan-TurboS

Hunyuan-TurboS is a large hybrid Mixture-of-Experts model that combines Mamba2 layers for long-sequence efficiency with grouped-query attention and feed-forward experts, activating 56B parameters out of 560B total across 128 layers arranged in an AMF/MF block pattern. It was pre-trained on 16T high-quality tokens and is positioned by its authors as the first industry-deployed large-scale Mamba-Transformer hybrid, with a native context reach of 256K that supports extended document and conversation workflows.

An adaptive long-short chain-of-thought mechanism lets the model answer simple queries quickly while reserving deeper deliberation for harder problems, with post-training that includes supervised fine-tuning on 3M instructions, a long-short CoT fusion method, multi-round deliberation learning, and a two-stage reinforcement learning pass covering STEM and general instruction following. Reported evaluations place it in the top tier of public chatbot arenas, with an LMSYS score of 1356 and an average of 77.9% across 23 automated benchmarks, suggesting a practical fit for reasoning-heavy and general assistant workloads where latency and cost efficiency matter.