AtomThink: Multimodal Slow Thinking with Atomic Step Reasoning

TL;DR

AtomThink introduces a novel multimodal reasoning framework that employs atomic step reasoning and structured chain of thought to enhance model accuracy, efficiency, and data utilization in complex tasks.

Contribution

The paper proposes AtomThink, a new multimodal reasoning paradigm with atomic steps, improving performance and efficiency over existing structured CoT methods.

Findings

Achieves over 10% accuracy improvement on MathVista and MathVerse datasets.

Enhances data utilization by 5 times compared to state-of-the-art methods.

Increases inference efficiency by 85.3%.

Abstract

In this paper, we address the challenging task of multimodal reasoning by incorporating the notion of ``slow thinking'' into multimodal large language models (MLLMs). Our core idea is that models can learn to adaptively use different levels of reasoning to tackle questions of varying complexity. We propose a novel paradigm of Self-structured Chain of Thought (SCoT), which consists of minimal semantic atomic steps. Unlike existing methods that rely on structured templates or free-form paradigms, our method not only generates flexible CoT structures for various complex tasks but also mitigates the phenomenon of overthinking for easier tasks. To introduce structured reasoning into visual cognition, we design a novel AtomThink framework with four key modules: (i) a data engine to generate high-quality multimodal reasoning paths; (ii) a supervised fine-tuning (SFT) process with serialized inference data; (iii) a policy-guided multi-turn inference method; and (iv) an atomic capability metric to evaluate the single-step utilization rate. Extensive experiments demonstrate that the proposed AtomThink significantly improves the performance of baseline MLLMs, achieving more than 10\% average accuracy gains on MathVista and MathVerse. Compared to state-of-the-art structured CoT approaches, our method not only achieves higher accuracy but also improves data utilization by 5 $\times$ and boosts inference efficiency by 85.3\%. Our code is publicly available at https://github.com/Kun-Xiang/AtomThink.