GRIT: Teaching MLLMs to Think with Images

TL;DR

GRIT is a novel training method for multimodal large language models that generates visually grounded reasoning chains by interleaving natural language and bounding box coordinates, improving reasoning and grounding capabilities.

Contribution

Introduces a grounded reasoning paradigm with a reinforcement learning approach that trains MLLMs to produce explicit visual reasoning without requiring annotated reasoning chains.

Findings

GRIT achieves high-quality, visually grounded reasoning with as few as 20 training triplets.

Models trained with GRIT produce coherent reasoning chains that are explicitly grounded in images.

GRIT enhances reasoning and grounding abilities in MLLMs effectively and data-efficiently.

Abstract

Recent studies have demonstrated the efficacy of using Reinforcement Learning (RL) in building reasoning models that articulate chains of thoughts prior to producing final answers. However, despite ongoing advances that aim at enabling reasoning for vision-language tasks, existing open-source visual reasoning models typically generate reasoning content with pure natural language, lacking explicit integration of visual information. This limits their ability to produce clearly articulated and visually grounded reasoning chains. To this end, we propose Grounded Reasoning with Images and Texts (GRIT), a novel method for training MLLMs to think with images. GRIT introduces a grounded reasoning paradigm, in which models generate reasoning chains that interleave natural language and explicit bounding box coordinates. These coordinates point to regions of the input image that the model consults during its reasoning process. Additionally, GRIT is equipped with a reinforcement learning approach, GRPO-GR, built upon the GRPO algorithm. GRPO-GR employs robust rewards focused on the final answer accuracy and format of the grounded reasoning output, which eliminates the need for data with reasoning chain annotations or explicit bounding box labels. As a result, GRIT achieves exceptional data efficiency, requiring as few as 20 image-question-answer triplets from existing datasets. Comprehensive evaluations demonstrate that GRIT effectively trains MLLMs to produce coherent and visually grounded reasoning chains, showing a successful unification of reasoning and grounding abilities.