Visual Agentic Reinforcement Fine-Tuning

TL;DR

This paper introduces Visual-ARFT, a reinforcement fine-tuning method that enhances large vision-language models with agentic abilities like web browsing and image code manipulation, evaluated on new benchmarks.

Contribution

The work presents Visual-ARFT, a novel fine-tuning approach that enables LVLMs to perform real-time web searches and image coding, advancing multi-modal agentic capabilities.

Findings

Outperforms baseline by +18.6% F1 / +13.0% EM on MAT-Coding

Achieves +29.3 F1% / +25.9% EM gains on multi-hop QA benchmarks

Surpasses GPT-4o in agentic multi-modal tasks

Abstract

A key trend in Large Reasoning Models (e.g., OpenAI's o3) is the native agentic ability to use external tools such as web browsers for searching and writing/executing code for image manipulation to think with images. In the open-source research community, while significant progress has been made in language-only agentic abilities such as function calling and tool integration, the development of multi-modal agentic capabilities that involve truly thinking with images, and their corresponding benchmarks, are still less explored. This work highlights the effectiveness of Visual Agentic Reinforcement Fine-Tuning (Visual-ARFT) for enabling flexible and adaptive reasoning abilities for Large Vision-Language Models (LVLMs). With Visual-ARFT, open-source LVLMs gain the ability to browse websites for real-time information updates and write code to manipulate and analyze input images through cropping, rotation, and other image processing techniques. We also present a Multi-modal Agentic Tool Bench (MAT) with two settings (MAT-Search and MAT-Coding) designed to evaluate LVLMs' agentic search and coding abilities. Our experimental results demonstrate that Visual-ARFT outperforms its baseline by +18.6% F1 / +13.0% EM on MAT-Coding and +10.3% F1 / +8.7% EM on MAT-Search, ultimately surpassing GPT-4o. Visual-ARFT also achieves +29.3 F1% / +25.9% EM gains on existing multi-hop QA benchmarks such as 2Wiki and HotpotQA, demonstrating strong generalization capabilities. Our findings suggest that Visual-ARFT offers a promising path toward building robust and generalizable multimodal agents.