No Data? No Problem: Robust Vision-Tabular Learning with Missing Values

TL;DR

RoVTL is a novel framework that enables robust multimodal learning from medical images and tabular data, effectively handling varying levels of missing tabular attributes during training and inference.

Contribution

It introduces a contrastive pretraining with tabular missingness augmentation and a gated cross-attention module for multimodal fusion, ensuring consistent performance across different data availability scenarios.

Findings

Outperforms prior methods in robustness to missing tabular data on cardiac MRI datasets.

Successfully generalizes to external datasets and natural images, demonstrating versatility.

Achieves stable performance regardless of the amount of available tabular information.

Abstract

Large-scale medical biobanks provide imaging data complemented by extensive tabular information, such as demographics or clinical measurements. However, this abundance of tabular attributes does not reflect real-world datasets, where only a subset of attributes may be available. This discrepancy calls for methods that can leverage all the tabular data during training while remaining robust to missing values at inference. To address this challenge, we propose RoVTL (Robust Vision-Tabular Learning), a framework designed to handle any level of tabular data availability, from 0% to 100%. RoVTL comprises two key stages: contrastive pretraining, where we introduce tabular attribute missingness as data augmentation to promote robustness, and downstream task tuning using a gated cross-attention module for multimodal fusion. During fine-tuning, we employ a novel Tabular More vs. Fewer loss that ranks performance based on the amount of available tabular data. Combined with disentangled gradient learning, this enables consistent performance across all tabular data completeness scenarios. We evaluate RoVTL on cardiac MRI scans from the UK Biobank, demonstrating superior robustness to missing tabular data compared to prior methods. Furthermore, RoVTL successfully generalizes to an external cardiac MRI dataset for multimodal disease classification, and extends to the natural images domain, achieving robust performance on a car advertisements dataset. The code is available at https://github.com/marteczkah/RoVTL.