mistral-small-4-119b-2603

Mistral Small 4 119B is positioned by its publisher as a single model that can behave as both a general instruction follower and a reasoning model, letting users toggle modes based on whether they need quick conversational answers or more deliberate analytical responses. It is shipped as an open-weight release, which makes it attractive for teams that want to self-host, fine-tune, or audit weights rather than depend on a closed API. The creator attribution is Mistral AI, with NVIDIA hosting a containerized distribution of the same checkpoint through its NGC catalog.

The model is built on a 36-layer transformer with a hidden size of 4,096, paired with multi-head latent attention and a mixture-of-experts feed-forward block that routes each token to four of 128 experts with one always active, balancing capacity against compute cost. This wider but sparsely activated design is what enables the hybrid behavior without paying full inference cost on every request. The tokenizer covers 131,072 entries and the architecture supports very long-context workloads, with the published architecture render describing windows up to 1,048,576 tokens for long-document analysis, code review, and multi-turn agentic flows that need to retain large histories.