Airoboros

Jon Durbin (jondurbin) · June 2023

activeOpen Weightdecoder onlytext
Context Window4K tokens
Variants7B, 13B, 70B

Description

Jon Durbin's uncensored model series, trained using synthetic data generated by GPT-4 through a self-instruct approach — where the AI generates its own training examples and progressively improves. Known for strong logical reasoning and unrestricted responses, and one of the earliest models to demonstrate that high-quality synthetic training data could produce excellent results.

Key Innovations

Instruction Tuning
Instruction TuningFine-tuning a model on instruction-response pairs so it follows user commands more reliably.
Open Weight
Open WeightModel weights are publicly released but training data/code may not be. Enables fine-tuning but not full reproduction.

Family Tree

Built On

Lineage

LLaMAAiroboros