Weakly-Supervised Multimodal Learning on MIMIC-CXR

TL;DR

This paper evaluates the Multimodal Variational Mixture-of-Experts VAE on MIMIC-CXR, showing it outperforms other models and fully supervised methods, addressing challenges in multimodal medical data integration and label scarcity.

Contribution

It introduces and thoroughly evaluates the MMVM VAE, demonstrating its superior performance in multimodal medical imaging tasks.

Findings

MMVM VAE outperforms other multimodal VAEs

MMVM VAE surpasses fully supervised approaches

Demonstrates potential for real-world medical applications

Abstract

Multimodal data integration and label scarcity pose significant challenges for machine learning in medical settings. To address these issues, we conduct an in-depth evaluation of the newly proposed Multimodal Variational Mixture-of-Experts (MMVM) VAE on the challenging MIMIC-CXR dataset. Our analysis demonstrates that the MMVM VAE consistently outperforms other multimodal VAEs and fully supervised approaches, highlighting its strong potential for real-world medical applications.