Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification

TL;DR

This paper introduces MVSL, a framework that enhances low-resource biomedical image classification by decoupling visual and textual adaptation, employing multi-granularity contrastive learning, and leveraging language models for semantic regularization.

Contribution

MVSL is a novel unified approach that improves parameter-efficient adaptation and fine-grained discrimination in biomedical image classification under limited supervision.

Findings

MVSL outperforms state-of-the-art methods on 11 biomedical datasets.

It achieves superior few-shot and zero-shot classification accuracy.

The framework effectively models both global and local image semantics.

Abstract

Accurate biomedical image classification under low-resource conditions remains challenging due to limited annotations, subtle inter-class visual differences, and complex disease semantics. While vision--language models offer a promising foundation for mitigating data scarcity, their effective adaptation in biomedical settings is constrained by the need for parameter-efficient tuning alongside fine-grained and semantically consistent representation learning. In this work, we propose Multi-View Synergistic Learning (MVSL), a unified framework that addresses these challenges by jointly considering adaptation paradigms, representation granularity, and disease semantic relationships. MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics, enabling more stable and effective parameter-efficient fine-tuning. It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence, improving fine-grained discrimination for visually similar disease categories. In addition, MVSL preserves disease-level semantic structure by incorporating structured supervision derived from large language models, which constrains textual representations at the class level and indirectly regularizes visual embeddings through cross-modal alignment. Together, these components enable more stable cross-modal alignment and improved discrimination under limited supervision. Extensive experiments on $11$ public biomedical datasets spanning $9$ imaging modalities and $10$ anatomical regions demonstrate that MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.