MMhops-R1: Multimodal Multi-hop Reasoning

TL;DR

This paper introduces MMhops, a large-scale benchmark for evaluating multi-modal multi-hop reasoning, and proposes MMhops-R1, a retrieval-augmented model that uses reinforcement learning for dynamic reasoning, significantly advancing multi-modal reasoning capabilities.

Contribution

The paper presents a new benchmark for multi-modal multi-hop reasoning and a novel retrieval-augmented framework that enhances dynamic reasoning and knowledge integration.

Findings

MMhops-R1 outperforms baselines on the MMhops benchmark.

Dynamic planning improves multi-hop reasoning performance.

The model generalizes well to fixed-hop reasoning tasks.

Abstract

The ability to perform multi-modal multi-hop reasoning by iteratively integrating information across various modalities and external knowledge is critical for addressing complex real-world challenges. However, existing Multi-modal Large Language Models (MLLMs) are predominantly limited to single-step reasoning, as existing benchmarks lack the complexity needed to evaluate and drive multi-hop abilities. To bridge this gap, we introduce MMhops, a novel, large-scale benchmark designed to systematically evaluate and foster multi-modal multi-hop reasoning. MMhops dataset comprises two challenging task formats, Bridging and Comparison, which necessitate that models dynamically construct complex reasoning chains by integrating external knowledge. To tackle the challenges posed by MMhops, we propose MMhops-R1, a novel multi-modal Retrieval-Augmented Generation (mRAG) framework for dynamic reasoning. Our framework utilizes reinforcement learning to optimize the model for autonomously planning reasoning paths, formulating targeted queries, and synthesizing multi-level information. Comprehensive experiments demonstrate that MMhops-R1 significantly outperforms strong baselines on MMhops, highlighting that dynamic planning and multi-modal knowledge integration are crucial for complex reasoning. Moreover, MMhops-R1 demonstrates strong generalization to tasks requiring fixed-hop reasoning, underscoring the robustness of our dynamic planning approach. In conclusion, our work contributes a challenging new benchmark and a powerful baseline model, and we will release the associated code, data, and weights to catalyze future research in this critical area.