MAmmoTH2: Scaling Instructions from the Web

TL;DR

This paper introduces MAmmoTH2, a method for efficiently harvesting 10 million instruction-response pairs from the web to improve large language models' reasoning abilities without relying on costly human annotation or GPT-4 data.

Contribution

It presents a novel paradigm for large-scale instruction data collection from web sources, significantly enhancing LLM reasoning performance.

Findings

MAmmoTH2-7B improves math reasoning accuracy from 11% to 36.7%.

MAmmoTH2-Plus achieves state-of-the-art results on reasoning benchmarks.

The approach reduces reliance on costly human or GPT-4 data for instruction tuning.

Abstract

Instruction tuning improves the reasoning abilities of large language models (LLMs), with data quality and scalability being the crucial factors. Most instruction tuning data come from human crowd-sourcing or GPT-4 distillation. We propose a paradigm to efficiently harvest 10 million naturally existing instruction data from the pre-training web corpus to enhance LLM reasoning. Our approach involves (1) recalling relevant documents, (2) extracting instruction-response pairs, and (3) refining the extracted pairs using open-source LLMs. Fine-tuning base LLMs on this dataset, we build MAmmoTH2 models, which significantly boost performance on reasoning benchmarks. Notably, MAmmoTH2-7B's (Mistral) performance increases from 11% to 36.7% on MATH and from 36% to 68.4% on GSM8K without training on any in-domain data. Further training MAmmoTH2 on public instruction tuning datasets yields MAmmoTH2-Plus, achieving state-of-the-art performance on several reasoning and chatbot benchmarks. Our work demonstrates how to harvest large-scale, high-quality instruction data without costly human annotation or GPT-4 distillation, providing a new paradigm for building better instruction tuning data.