From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models

TL;DR

This paper systematically reviews Multimodal Chain-of-Thought methods, analyzing their mechanisms, benchmarks, challenges, and future directions to enhance reasoning in multimodal large language models.

Contribution

It provides a comprehensive analysis of MCoT approaches, including paradigms, training, inference, and evaluation, highlighting their theoretical foundations and practical applications.

Findings

MCoT improves reasoning transparency in multimodal models

Evaluation benchmarks for MCoT are summarized and analyzed

Challenges in MCoT include generalization and interpretability

Abstract

With the remarkable success of Multimodal Large Language Models (MLLMs) in perception tasks, enhancing their complex reasoning capabilities has emerged as a critical research focus. Existing models still suffer from challenges such as opaque reasoning paths and insufficient generalization ability. Chain-of-Thought (CoT) reasoning, which has demonstrated significant efficacy in language models by enhancing reasoning transparency and output interpretability, holds promise for improving model reasoning capabilities when extended to the multimodal domain. This paper provides a systematic review centered on "Multimodal Chain-of-Thought" (MCoT). First, it analyzes the background and theoretical motivations for its inception from the perspectives of technical evolution and task demands. Then, it introduces mainstream MCoT methods from three aspects: CoT paradigms, the post-training stage, and the inference stage, while also analyzing their underlying mechanisms. Furthermore, the paper summarizes existing evaluation benchmarks and metrics, and discusses the application scenarios of MCoT. Finally, it analyzes the challenges currently facing MCoT and provides an outlook on its future research directions.