The Impact of Image Resolution on Biomedical Multimodal Large Language Models

TL;DR

This paper investigates how image resolution impacts the performance of biomedical multimodal large language models, emphasizing the importance of native-resolution training and inference for optimal results.

Contribution

It reveals that native-resolution training and inference significantly enhance biomedical MLLM performance and proposes mixed-resolution training as a practical solution for real-world applications.

Findings

Native-resolution training improves performance

Resolution misalignment degrades results

Mixed-resolution training mitigates performance loss

Abstract

Imaging technologies are fundamental to biomedical research and modern medicine, requiring analysis of high-resolution images across various modalities. While multimodal large language models (MLLMs) show promise for biomedical image analysis, most are designed for low-resolution images from general-purpose datasets, risking critical information loss. We investigate how image resolution affects MLLM performance in biomedical applications and demonstrate that: (1) native-resolution training and inference significantly improve performance across multiple tasks, (2) misalignment between training and inference resolutions severely degrades performance, and (3) mixed-resolution training effectively mitigates misalignment and balances computational constraints with performance requirements. Based on these findings, we recommend prioritizing native-resolution inference and mixed-resolution datasets to optimize biomedical MLLMs for transformative impact in scientific research and clinical applications.