MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning

TL;DR

This paper introduces MedVistaGym, a training environment that enhances medical vision language models by enabling multi-step, tool-integrated reasoning for medical image analysis, significantly improving performance on medical VQA benchmarks.

Contribution

We present MedVistaGym, a scalable, interactive training environment that trains models to effectively select and use tools for multi-modal medical reasoning, a capability lacking in open-source VLMs.

Findings

MedVistaGym-R1-8B outperforms baselines by 19.10% to 24.21% on six medical VQA benchmarks.

Structured agentic training significantly improves tool-integrated reasoning.

The environment enables models to localize relevant image regions and coordinate tool use effectively.

Abstract

Vision language models (VLMs) achieve strong performance on general image understanding but struggle to think with medical images, especially when performing multi-step reasoning through iterative visual interaction. Medical VLMs often rely on static visual embeddings and single-pass inference, preventing models from re-examining, verifying, or refining visual evidence during reasoning. While tool-integrated reasoning offers a promising path forward, open-source VLMs lack the training infrastructure to learn effective tool selection, invocation, and coordination in multi-modal medical reasoning. We introduce MedVistaGym, a scalable and interactive training environment that incentivizes tool-integrated visual reasoning for medical image analysis. MedVistaGym equips VLMs to determine when and which tools to invoke, localize task-relevant image regions, and integrate single or multiple sub-image evidence into interleaved multimodal reasoning within a unified, executable interface for agentic training. Using MedVistaGym, we train MedVistaGym-R1 to interleave tool use with agentic reasoning through trajectory sampling and end-to-end reinforcement learning. Across six medical VQA benchmarks, MedVistaGym-R1-8B exceeds comparably sized tool-augmented baselines by 19.10% to 24.21%, demonstrating that structured agentic training--not tool access alone--unlocks effective tool-integrated reasoning for medical image analysis.