VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learning

TL;DR

VG-Refiner introduces a two-stage think-rethink mechanism with refinement rewards to improve tool-based visual reasoning, effectively handling unreliable tool outputs and enhancing grounding accuracy.

Contribution

It is the first framework for tool-refined referring grounded reasoning, incorporating explicit analysis, response mechanisms, and evaluation metrics for refinement ability.

Findings

Significant accuracy improvements on grounding benchmarks.

Enhanced correction ability in referring tasks.

Effective handling of unreliable tool outputs.

Abstract

Tool-integrated visual reasoning (TiVR) has demonstrated great potential in enhancing multimodal problem-solving. However, existing TiVR paradigms mainly focus on integrating various visual tools through reinforcement learning, while neglecting to design effective response mechanisms for handling unreliable or erroneous tool outputs. This limitation is particularly pronounced in referring and grounding tasks, where inaccurate detection tool predictions often mislead TiVR models into generating hallucinated reasoning. To address this issue, we propose the VG-Refiner, the first framework aiming at the tool-refined referring grounded reasoning. Technically, we introduce a two-stage think-rethink mechanism that enables the model to explicitly analyze and respond to tool feedback, along with a refinement reward that encourages effective correction in response to poor tool results. In addition, we propose two new metrics and establish fair evaluation protocols to systematically measure the refinement ability of current models. We adopt a small amount of task-specific data to enhance the refinement capability of VG-Refiner, achieving a significant improvement in accuracy and correction ability on referring and reasoning grounding benchmarks while preserving the general capabilities of the pretrained model.