Grok-1
xAI · November 2023
◌ legacyOpen Weightmixture of expertstext
Parameters314B (MoE)
Context Window8K tokens
Why It Matters
Largest open-source language model at time of release at 314B parameters. Pioneered real-time social media integration for conversational AI.
Description
xAI's debut model with 314 billion parameters using a Mixture-of-Experts architecture — the largest model to have its weights publicly released at the time. Built with real-time access to posts on X (formerly Twitter), giving it up-to-the-minute knowledge that other models lacked. Known for its irreverent, humor-forward personality.
Notable Milestones
- ▸Largest openly released model at launch (314B parameters)
- ▸Real-time access to X/Twitter posts
- ▸Open-sourced weights under Apache 2.0 license
Key Innovations
MoE
MoEArchitecture where only a fraction of the model's parameters are active for each input, allowing massive scale with lower compute.
Open Weight
Open WeightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction.