Qwen3 Next 80B A3B Thinking

Qwen3 Next 80B A3B Thinking is a reasoning-first model built on the innovative Qwen3-Next architecture, which prioritizes extreme efficiency in both training and inference. The model utilizes a hybrid attention mechanism that combines Gated DeltaNet and Gated Attention to manage long-context modeling effectively. By employing a high-sparsity Mixture-of-Experts structure, it achieves a low activation ratio, requiring only 3 billion parameters during inference despite its 80-billion-parameter scale. This design, complemented by multi-token prediction, allows the model to deliver high throughput and rapid generation, making it well-suited for demanding applications like code synthesis, mathematical proofs, and complex logic.

The model benefits from advanced stability optimizations, including zero-centered and weight-decayed layernorm, which ensure robust performance during pre-training and post-training. Through the application of GSPO, the developers successfully addressed the stability challenges inherent in combining hybrid attention with high-sparsity MoE architectures during reinforcement learning. As a reasoning-enhanced variant, it is specifically tuned to output structured thinking traces, providing a reliable foundation for agentic frameworks, tool use, and retrieval-heavy workflows that require detailed, step-by-step reasoning while maintaining instruction adherence and reducing off-task behavior.