HyPCA-Net: Advancing Multimodal Fusion in Medical Image Analysis

TL;DR

HyPCA-Net introduces a novel multimodal fusion network for medical image analysis that enhances shared representation learning while reducing computational costs, outperforming existing methods across multiple datasets.

Contribution

The paper presents HyPCA-Net, a new efficient multimodal fusion architecture with residual adaptive learning and dual-view cascaded attention blocks for improved medical image analysis.

Findings

Achieves up to 5.2% performance improvement over existing methods.

Reduces computational cost by up to 73.1%.

Outperforms state-of-the-art on ten datasets.

Abstract

Multimodal fusion frameworks, which integrate diverse medical imaging modalities (e.g., MRI, CT), have shown great potential in applications such as skin cancer detection, dementia diagnosis, and brain tumor prediction. However, existing multimodal fusion methods face significant challenges. First, they often rely on computationally expensive models, limiting their applicability in low-resource environments. Second, they often employ cascaded attention modules, which potentially increase risk of information loss during inter-module transitions and hinder their capacity to effectively capture robust shared representations across modalities. This restricts their generalization in multi-disease analysis tasks. To address these limitations, we propose a Hybrid Parallel-Fusion Cascaded Attention Network (HyPCA-Net), composed of two core novel blocks: (a) a computationally efficient residual adaptive learning attention block for capturing refined modality-specific representations, and (b) a dual-view cascaded attention block aimed at learning robust shared representations across diverse modalities. Extensive experiments on ten publicly available datasets exhibit that HyPCA-Net significantly outperforms existing leading methods, with improvements of up to 5.2% in performance and reductions of up to 73.1% in computational cost. Code: https://github.com/misti1203/HyPCA-Net.