Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation

TL;DR

This paper introduces a personalized invariant representation learning framework to improve cross-modality generalization in multi-modal medical imaging, addressing individual differences often overlooked by existing methods.

Contribution

It proposes a two-stage approach combining invariant representation pre-training with fine-tuning, backed by theoretical and empirical evidence demonstrating improved generalization.

Findings

Enhanced multi-modal generalization performance

Superior transferability across diverse tasks

Significant improvements over non-personalized methods

Abstract

Variations in medical imaging modalities and individual anatomical differences pose challenges to cross-modality generalization in multi-modal tasks. Existing methods often concentrate exclusively on common anatomical patterns, thereby neglecting individual differences and consequently limiting their generalization performance. This paper emphasizes the critical role of learning individual-level invariance, i.e., personalized representation $\mathbb{X}_h$, to enhance multi-modality generalization under both homogeneous and heterogeneous settings. It reveals that mappings from individual biological profile to different medical modalities remain static across the population, which is implied in the personalization process. We propose a two-stage approach: pre-training with invariant representation $\mathbb{X}_h$ for personalization, then fine-tuning for diverse downstream tasks. We provide both theoretical and empirical evidence demonstrating the feasibility and advantages of personalization, showing that our approach yields greater generalizability and transferability across diverse multi-modal medical tasks compared to methods lacking personalization. Extensive experiments further validate that our approach significantly enhances performance in various generalization scenarios.