Qwen: Qwen3 Next 80B A3B Thinking

This model represents a significant evolution in foundation architecture, utilizing a high-sparsity mixture-of-experts design that activates only 3 billion of its 80 billion parameters during inference. By replacing standard attention with a hybrid mechanism combining Gated DeltaNet and Gated Attention, it achieves efficient context modeling for long-sequence tasks. The design intent focuses on extreme scaling efficiency, drastically reducing the computational cost per token while maintaining the capacity required for complex, agentic workflows. It is specifically engineered to excel at multi-step reasoning, including mathematical proofs, code synthesis, and logic-heavy planning.

The model benefits from a robust training lineage that incorporates multi-token prediction to accelerate inference and stability-focused techniques like zero-centered, weight-decayed layer normalization. To manage the complexities of its hybrid attention and sparse architecture, the development team utilized specialized reinforcement learning methods to ensure stability during post-training. These advancements allow the model to produce structured thinking traces, making it a strong candidate for retrieval-heavy workflows and agent frameworks that demand detailed, step-by-step problem solving. Its architecture is optimized for high throughput, providing a practical balance between deep reasoning capabilities and operational efficiency.