Qwen
Alibaba Cloud · August 2023
● activeOpen Weightdecoder onlytext
Parameters7B - 72B
Context Window32K tokens
Variants7B, 14B, 72B
Why It Matters
Alibaba's entry into the open-weight LLM race, establishing China's largest tech company as a serious competitor in foundation models.
Description
Alibaba Cloud's first large language model family, available in sizes from 7B to 72B parameters. Designed with strong bilingual capabilities in Chinese and English, making it particularly useful for businesses operating across both languages. Trained on 3 trillion tokens spanning multiple languages and domains.
Key Innovations
Open Weight
Open WeightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction.
Family Tree
Successors (1)
Related Research (1)
RoPEArchitecture
2021 · Zhuiyi Technology
Introduced rotary position embeddings that encode position via rotation matrices, enabling better length generalization. Used by virtually every moder…