Phi-4
Microsoft Research · February 2025
● activeOpen Sourcedecoder onlytext
Parameters14B
Context Window16K tokens
VariantsPhi-4 (14B), Phi-4-mini (3.8B), Phi-4-multimodal (5.6B), Phi-4-reasoning
Why It Matters
Proved that small models could compete with frontier models on complex STEM reasoning, and expanded the Phi line into multimodal and dedicated reasoning capabilities.
Description
A 14 billion parameter model that pushed small models into advanced reasoning territory, particularly excelling at math, science, and STEM problems. Later expanded with multimodal variants that can process speech, images, and text, and dedicated reasoning models that use structured 'thinking' steps to solve complex problems.
Notable Milestones
- ▸Advanced math and science problem solving
- ▸Multimodal reasoning across speech, vision, and text
- ▸Competed with much larger reasoning models on STEM benchmarks
Benchmark Scores
MMLUMassive Multitask Language Understanding — 57 subjects
84.8%HumanEvalCode generation pass@1 — Python problems
82.6%Key Innovations
Reasoning
ReasoningStructured step-by-step problem solving, often using chain-of-thought or tree-of-thought approaches.
Distillation
DistillationTraining a smaller 'student' model to mimic a larger 'teacher' model, preserving capability at lower cost.