Phi-4-mini

Phi-4-mini is a compact 3.8B-parameter dense decoder-only Transformer that emerged from Microsoft's effort to prove smaller models can punch above their weight. The model was trained on a carefully curated blend of synthetic data and filtered publicly available websites, with a deliberate emphasis on reasoning-dense material. This architectural lineage represents a deliberate move away from the assumption that raw scale determines capability. Key engineering choices include a 200K vocabulary, grouped-query attention, and shared embedding layers, all of which work together to maximize performance within a tight parameter budget.

The model underwent a post-training enhancement combining supervised fine-tuning and direct preference optimization, a combination that sharpens instruction adherence while reinforcing safety guardrails. Trained on data through June 2024, the model was designed for real-world deployment in settings where memory and compute are limited, and latency matters. Its ability to outperform larger models in reasoning, math, coding, and function-calling makes it particularly well-suited for developers building AI-powered features in constrained environments. With broad multilingual support spanning dozens of languages, the model serves both commercial and research communities as a versatile, efficient building block for generative AI applications.