Synthetic Privileged Information Enhances Medical Image Representation Learning

TL;DR

This paper introduces a method that uses synthetic paired data generated from unpaired datasets to improve medical image representation learning, outperforming traditional multi-modal and single-modality training.

Contribution

The novel approach leverages synthetic privileged information to enhance representation learning, reducing errors significantly compared to existing methods.

Findings

Up to 4.4x error reduction over single-modality training

Up to 5.6x error reduction compared to authentic multi-modal datasets

Synthetic data improves learning even with small datasets

Abstract

Multimodal self-supervised representation learning has consistently proven to be a highly effective method in medical image analysis, offering strong task performance and producing biologically informed insights. However, these methods heavily rely on large, paired datasets, which is prohibitive for their use in scenarios where paired data does not exist, or there is only a small amount available. In contrast, image generation methods can work well on very small datasets, and can find mappings between unpaired datasets, meaning an effectively unlimited amount of paired synthetic data can be generated. In this work, we demonstrate that representation learning can be significantly improved by synthetically generating paired information, both compared to training on either single-modality (up to 4.4x error reduction) or authentic multi-modal paired datasets (up to 5.6x error reduction).