FLAN-T5
Google · October 2022
● activeOpen Sourceencoder decodertext
Why It Matters
Showed that instruction tuning at scale could make smaller models competitive with much larger ones, influencing the development of efficient fine-tuning techniques across the industry.
Description
Google's instruction-tuned version of T5, fine-tuned on over 1,800 tasks described via natural language instructions. Demonstrated that instruction tuning dramatically improves zero-shot and few-shot performance across virtually all NLP tasks.
Key Innovations
Instruction Tuning
Instruction TuningFine-tuning a model on instruction-response pairs so it follows user commands more reliably.
Few-Shot
Few-ShotLearning from just a handful of examples provided in the prompt, without retraining.