DeepSeek V4 Pro is built around a Mixture-of-Experts design with 1.6 trillion total parameters while activating roughly 49 billion parameters per token, paired with a million-token context window. Its hybrid attention architecture combines Compressed Sparse Attention and Heavily Compressed Attention to keep long-context inference affordable, and Manifold-Constrained Hyper-Connections stabilize the very deep residual stream. Training used the Muon optimizer over more than 32 trillion tokens, giving the family a systems-engineering posture rather than a pure scale-up story, and DSpark speculative decoding further accelerates autoregressive generation in served deployments.
In practical terms, V4 Pro targets long-horizon reasoning and repository-scale software work, and independent evaluations show where it genuinely leads and where it falls short. It scored 96.4% on SWE-bench Verified (the top open-weight result in that measurement) and led open-weight models on the GDPval-AA real-world task benchmark, yet it trailed closed frontier systems on broader reasoning composites and showed uneven results on sandboxed agentic workflows. MIT-licensed open weights make it suitable for self-hosted pipelines, while gateway routes offer immediate access for teams that want the long-context, coding, and tool-calling strengths without standing up multi-GPU clusters.
