Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions

TL;DR

Marco-o1 advances open reasoning models capable of handling open-ended, complex real-world problems by integrating Chain-of-Thought fine-tuning, Monte Carlo Tree Search, and reflection mechanisms, extending reasoning beyond standard-answer domains.

Contribution

It introduces Marco-o1, a reasoning model that combines multiple strategies to generalize effectively to open-ended, less structured domains.

Findings

Effective in open-ended reasoning tasks

Outperforms baseline models in complex problem-solving

Demonstrates adaptability to real-world scenarios

Abstract

Currently OpenAI o1 sparks a surge of interest in the study of large reasoning models (LRM). Building on this momentum, Marco-o1 not only focuses on disciplines with standard answers, such as mathematics, physics, and coding -- which are well-suited for reinforcement learning (RL) -- but also places greater emphasis on open-ended resolutions. We aim to address the question: ''Can the o1 model effectively generalize to broader domains where clear standards are absent and rewards are challenging to quantify?'' Marco-o1 is powered by Chain-of-Thought (CoT) fine-tuning, Monte Carlo Tree Search (MCTS), reflection mechanisms, and innovative reasoning strategies -- optimized for complex real-world problem-solving tasks.