Qwen 2.5
Alibaba Cloud · September 2024
● activeOpen Sourcedecoder onlytext
Parameters0.5B - 72B
Context Window128K tokens
VariantsQwen2.5, Qwen2.5-Coder, Qwen2.5-Math
Why It Matters
Became the most downloaded open-source LLM globally, overtaking Meta's LLaMA. Its Apache 2.0 license and specialized coding/math variants made it the go-to base model for the open-source community.
Description
Became the most downloaded open-source LLM in the world, surpassing even Meta's LLaMA. Available in sizes from 0.5B to 72B with specialized variants for coding and mathematics. Released under the permissive Apache 2.0 license, allowing anyone to use it freely — even commercially.
Notable Milestones
- ▸Most downloaded open-source LLM on Hugging Face
- ▸Base model for DeepSeek R1 distilled variants
- ▸Best-in-class open-source coding model
Benchmark Scores
MMLUMassive Multitask Language Understanding — 57 subjects
86.8%HumanEvalCode generation pass@1 — Python problems
86.6%MATHMATH benchmark — competition-level problems
83.1%Key Innovations
Open Weight
Open WeightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction.
Code Gen
Code GenAbility to write, debug, and understand programming code across multiple languages.
Reasoning
ReasoningStructured step-by-step problem solving, often using chain-of-thought or tree-of-thought approaches.
Related Research (1)
RoPEArchitecture
2021 · Zhuiyi Technology
Introduced rotary position embeddings that encode position via rotation matrices, enabling better length generalization. Used by virtually every moder…