MMSearch-R1: Incentivizing LMMs to Search

TL;DR

This paper introduces MMSearch-R1, an end-to-end reinforcement learning framework that enables large multimodal models to perform efficient, on-demand multi-turn searches in real-world internet environments, improving search efficiency and performance.

Contribution

The paper presents the first reinforcement learning framework for multimodal search, integrating image and text tools, and introduces a new multimodal search VQA dataset for training and evaluation.

Findings

Outperforms RAG-based baselines of the same size.

Matches larger RAG models' performance with 30% fewer search calls.

Provides insights into efficient multimodal search behavior.

Abstract

Robust deployment of large multimodal models (LMMs) in real-world scenarios requires access to external knowledge sources, given the complexity and dynamic nature of real-world information. Existing approaches such as retrieval-augmented generation (RAG) and prompt engineered search agents rely on rigid pipelines, often leading to inefficient or excessive search behaviors. We present MMSearch-R1, the first end-to-end reinforcement learning framework that enables LMMs to perform on-demand, multi-turn search in real-world Internet environments. Our framework integrates both image and text search tools, allowing the model to reason about when and how to invoke them guided by an outcome-based reward with a search penalty. To support training, We collect a multimodal search VQA dataset through a semi-automated pipeline that covers diverse visual and textual knowledge needs and curate a search-balanced subset with both search-required and search-free samples, which proves essential for shaping efficient and on-demand search behavior. Extensive experiments on knowledge-intensive and info-seeking VQA tasks show that our model not only outperforms RAG-based baselines of the same model size, but also matches the performance of a larger RAG-based model while reducing search calls by over 30%. We further analyze key empirical findings to offer actionable insights for advancing research in multimodal search.