GenQA: Generating Millions of Instructions from a Handful of Prompts

TL;DR

This paper introduces GenQA, a method for automatically generating large-scale instruction datasets from minimal prompts, enabling effective fine-tuning of language models with minimal human effort.

Contribution

The authors present a scalable, automated approach to create extensive instruction datasets from a single prompt, improving fine-tuning outcomes for language models.

Findings

Generated datasets outperform WizardLM and Ultrachat on knowledge tasks.

Fine-tuned Llama-3 8B achieves competitive results on benchmarks.

Automated data creation reduces human oversight significantly.

Abstract

Most public instruction finetuning datasets are relatively small compared to the closed source datasets used to train industry models. To study questions about finetuning at scale, such as curricula and learning rate cooldown schedules, there is a need for industrial-scale datasets. However, this scale necessitates a data generation process that is almost entirely automated. In this work, we study methods for generating large instruction datasets from a single prompt. With little human oversight, we get LLMs to write diverse sets of instruction examples ranging from simple completion tasks to complex multi-turn dialogs across a variety of subject areas. When finetuning a Llama-3 8B base model, our dataset meets or exceeds both WizardLM and Ultrachat on both knowledge-intensive leaderboard tasks as well as conversational evaluations. We release our dataset, the "generator" prompts that created it, and our finetuned model checkpoints.