MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling

TL;DR

MedScope introduces a tool-using, coarse-to-fine reasoning framework for clinical videos, significantly improving the accuracy and trustworthiness of medical video understanding by explicitly grounding predictions in temporally localized visual evidence.

Contribution

This work presents MedScope, a novel model that performs iterative, evidence-guided reasoning with tool calls for clinical videos, and introduces ClinVideoSuite for high-fidelity supervision.

Findings

Achieves state-of-the-art performance on clinical video benchmarks.

Effectively grounds predictions in temporally localized visual evidence.

Outperforms existing models in both in-domain and out-of-domain evaluations.

Abstract

Long-form clinical videos are central to visual evidence-based decision-making, with growing importance for applications such as surgical robotics and related settings. However, current multimodal large language models typically process videos with passive sampling or weakly grounded inspection, which limits their ability to iteratively locate, verify, and justify predictions with temporally targeted evidence. To close this gap, we propose MedScope, a tool-using clinical video reasoning model that performs coarse-to-fine evidence seeking over long-form procedures. By interleaving intermediate reasoning with targeted tool calls and verification on retrieved observations, MedScope produces more accurate and trustworthy predictions that are explicitly grounded in temporally localized visual evidence. To address the lack of high-fidelity supervision, we build ClinVideoSuite, an evidence-centric, fine-grained clinical video suite. We then optimize MedScope with Grounding-Aware Group Relative Policy Optimization (GA-GRPO), which directly reinforces tool use with grounding-aligned rewards and evidence-weighted advantages. On full and fine-grained video understanding benchmarks, MedScope achieves state-of-the-art performance in both in-domain and out-of-domain evaluations. Our approach illuminates a path toward medical AI agents that can genuinely "think with videos" through tool-integrated reasoning. We will release our code, models, and data.