InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning

TL;DR

InfiMed introduces a novel training approach combining high-quality textual reasoning data, synthetic reflective CoT, and RLVR to enhance multimodal medical language models, achieving state-of-the-art results on medical benchmarks.

Contribution

The paper presents InfiMed, a new framework that improves medical multimodal LLMs by integrating diverse data and reasoning techniques, surpassing existing models in accuracy and reasoning ability.

Findings

InfiMed-RL-3B outperforms larger models like InternVL3-8B in medical benchmarks.

Incorporating synthetic reflective CoT enhances reasoning capabilities.

Training with 188K SFT samples and 36K RLVR samples yields superior performance.

Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in domains such as visual understanding and mathematical reasoning. However, their application in the medical domain is constrained by two key challenges: (1) multimodal medical datasets are scarce and often contain sparse information, limiting reasoning depth; and (2) Reinforcement Learning with Verifiable Rewards (RLVR), though effective in general domains, cannot reliably improve model performance in the medical domain. To overcome these challenges, during the supervised fine-tuning (SFT) stage, we incorporate high-quality textual reasoning data and general multimodal data alongside multimodal medical data to efficiently enhance foundational medical capabilities and restore the base model's reasoning ability. Moreover, considering that there are some multimodal medical datasets with sparse information, we further synthesize reflective-pattern-injected chain-of-thought (CoT) in addition to general CoT samples, equipping the model with initial reflective reasoning capabilities that provide a structured foundation for subsequent RLVR training. Finally, we introduce our InfiMed-Series models, InfiMed-SFT-3B and InfiMed-RL-3B, both of which deliver state-of-the-art performance across seven multimodal medical benchmarks. Notably, InfiMed-RL-3B achieves an average accuracy of 59.2%, outperforming even larger models like InternVL3-8B, which achieves 57.3%. Specifically, during the SFT phase, we utilized 188K samples, while the RLVR phase incorporated 36K samples, demonstrating the efficacy of both training strategies in achieving superior performance. We also conducted a series of extensive experiments, which provide valuable insights that contribute to advancing the performance of MLLMs in medical scenarios.