Dealing with missing data using attention and latent space regularization

TL;DR

This paper introduces a novel theoretical framework for handling missing data directly through observed variables, utilizing attention mechanisms and latent space regularization, avoiding imputation and improving robustness.

Contribution

It develops a measure-theoretic approach for modeling incomplete datasets without imputation, demonstrating theoretical properties and empirical superiority over existing methods.

Findings

Outperforms state-of-the-art models on benchmark datasets

Effectively handles various missingness patterns

Reduces bias without data imputation

Abstract

Most practical data science problems encounter missing data. A wide variety of solutions exist, each with strengths and weaknesses that depend upon the missingness-generating process. Here we develop a theoretical framework for training and inference using only observed variables enabling modeling of incomplete datasets without imputation. Using an information and measure-theoretic argument we construct models with latent space representations that regularize against the potential bias introduced by missing data. The theoretical properties of this approach are demonstrated empirically using a synthetic dataset. The performance of this approach is tested on 11 benchmarking datasets with missingness and 18 datasets corrupted across three missingness patterns with comparison against a state-of-the-art model and industry-standard imputation. We show that our proposed method overcomes the weaknesses of imputation methods and outperforms the current state-of-the-art.