NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions

TL;DR

NaturalReasoning introduces a large, diverse dataset of 2.8 million reasoning questions across multiple domains, enabling improved reasoning capabilities and self-training for AI models.

Contribution

It presents a scalable method for generating high-quality reasoning questions and releases a comprehensive dataset for advancing reasoning research.

Findings

Effective knowledge distillation from strong teacher models.

Successful unsupervised self-training using external reward models.

Demonstrated diversity and challenge level of the dataset.

Abstract

Scaling reasoning capabilities beyond traditional domains such as math and coding is hindered by the lack of diverse and high-quality questions. To overcome this limitation, we introduce a scalable approach for generating diverse and challenging reasoning questions, accompanied by reference answers. We present NaturalReasoning, a comprehensive dataset comprising 2.8 million questions that span multiple domains, including STEM fields (e.g., Physics, Computer Science), Economics, Social Sciences, and more. We demonstrate the utility of the questions in NaturalReasoning through knowledge distillation experiments which show that NaturalReasoning can effectively elicit and transfer reasoning capabilities from a strong teacher model. Furthermore, we demonstrate that NaturalReasoning is also effective for unsupervised self-training using external reward models or self-rewarding. To foster future work, we publicly release NaturalReasoning at https://huggingface.co/datasets/facebook/natural_reasoning.