Nemotron 3 Super 120B A12B (Nebius)

Nemotron 3 Super 120B A12B is an NVIDIA-built large language model aimed at agentic, reasoning, and conversational tasks. Its defining architectural choice is a hybrid Latent Mixture-of-Experts layout that interleaves Mamba-2 layers with MoE layers, paired with Multi-Token Prediction to accelerate generation. The result is a model that carries 120 billion total parameters but only activates 12 billion per forward pass, a sparse activation pattern that aims to balance raw capacity with inference efficiency rather than running every weight on every token.

Beyond the core architecture, the model is documented for multilingual use across English, French, German, Italian, Japanese, Spanish, and Chinese, and it is already being deployed by independent inference providers such as DeepInfra and Fireworks AI, the latter offering an NVFP4 quantized variant for on-demand serving. That third-party hosting footprint suggests practical availability across latency-sensitive and cost-sensitive production scenarios, and the multi-token prediction design is positioned to deliver faster token throughput than comparable dense checkpoints. It is a reasonable fit for teams that want strong open-weight reasoning and tool-using behavior without paying the full inference cost of a dense 120B model, especially when agentic pipelines and long-context reasoning are central requirements.