O1 Replication Journey: A Strategic Progress Report -- Part 1

TL;DR

This paper documents the real-time, transparent replication of OpenAI's O1 model, introduces the journey learning paradigm, and demonstrates its superior performance with minimal data, fostering open science and collective progress.

Contribution

It presents the innovative journey learning paradigm and a comprehensive, transparent approach to AI research replication, emphasizing open science and community engagement.

Findings

Journey learning outperformed supervised learning by over 8% on MATH dataset.

Real-time documentation and transparency foster open science and community collaboration.

Proposed methods require only 327 samples without additional tricks.

Abstract

This paper introduces a pioneering approach to artificial intelligence research, embodied in our O1 Replication Journey. In response to the announcement of OpenAI's groundbreaking O1 model, we embark on a transparent, real-time exploration to replicate its capabilities while reimagining the process of conducting and communicating AI research. Our methodology addresses critical challenges in modern AI research, including the insularity of prolonged team-based projects, delayed information sharing, and the lack of recognition for diverse contributions. By providing comprehensive, real-time documentation of our replication efforts, including both successes and failures, we aim to foster open science, accelerate collective advancement, and lay the groundwork for AI-driven scientific discovery. Our research progress report diverges significantly from traditional research papers, offering continuous updates, full process transparency, and active community engagement throughout the research journey. Technologically, we proposed the journey learning paradigm, which encourages models to learn not just shortcuts, but the complete exploration process, including trial and error, reflection, and backtracking. With only 327 training samples and without any additional tricks, journey learning outperformed conventional supervised learning by over 8\% on the MATH dataset, demonstrating its extremely powerful potential. We believe this to be the most crucial component of O1 technology that we have successfully decoded. We share valuable resources including technical hypotheses and insights, cognitive exploration maps, custom-developed tools, etc at https://github.com/GAIR-NLP/O1-Journey.