Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models

TL;DR

Med-Evo introduces a novel self-evolution framework for medical multimodal large language models that leverages unlabeled test data through label-free reinforcement learning, enhancing performance without additional labeled data.

Contribution

This paper presents Med-Evo, the first self-evolution approach for medical MLLMs utilizing label-free reinforcement learning with innovative pseudo labeling and hierarchical reward mechanisms.

Findings

Achieved 10.43% accuracy improvement on SLAKE dataset

Demonstrated effectiveness across three medical VQA benchmarks

Outperformed state-of-the-art methods significantly

Abstract

Medical Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across diverse healthcare tasks. However, current post-training strategies, such as supervised fine-tuning and reinforcement learning, heavily depend on substantial annotated data while overlooking the potential of unlabeled test data for model enhancement. This limitation becomes particularly pronounced in medical domains, where acquiring extensive labeled medical data is difficult due to the strict data sensitivity and annotation complexity. Moreover, leveraging test data poses challenges in generating reliable supervision signals from unlabeled samples and maintaining stable self-evolution. To address these limitations, we propose Med-Evo, the first self-evolution framework for medical MLLMs that utilizes label-free reinforcement learning to promote model performance without requiring additional labeled data. Our framework introduces two key innovations: $1)$ Feature-driven Pseudo Labeling (FPL) that identifies semantic centroids from all heterogeneous candidate responses to select pseudo labels in each rollout, and $2)$ Hard-Soft Reward (HSR) that combines exact match with token-level assessment and semantic similarity to provide hierarchical reward. Experiments on three medical VQA benchmarks and two base MLLMs show clear advantages of our approach over SOTA methods, with significant improvements of 10.43\% accuracy and 4.68\% recall on the SLAKE dataset using Qwen2.5-VL, showing the effectiveness of our method.