MIWAE: Deep Generative Modelling and Imputation of Incomplete Data

TL;DR

MIWAE introduces a deep generative modeling technique that effectively handles missing data during training and provides accurate imputations without additional computational costs, demonstrating competitive performance on image and tabular datasets.

Contribution

MIWAE extends the IWAE framework to efficiently train deep latent variable models with missing-at-random data, enabling accurate imputation and competitive performance.

Findings

MIWAE achieves accurate imputations on various datasets.

Training on incomplete data yields similar test performance to complete data.

MIWAE outperforms or matches state-of-the-art imputation methods.

Abstract

We consider the problem of handling missing data with deep latent variable models (DLVMs). First, we present a simple technique to train DLVMs when the training set contains missing-at-random data. Our approach, called MIWAE, is based on the importance-weighted autoencoder (IWAE), and maximises a potentially tight lower bound of the log-likelihood of the observed data. Compared to the original IWAE, our algorithm does not induce any additional computational overhead due to the missing data. We also develop Monte Carlo techniques for single and multiple imputation using a DLVM trained on an incomplete data set. We illustrate our approach by training a convolutional DLVM on a static binarisation of MNIST that contains 50% of missing pixels. Leveraging multiple imputation, a convolutional network trained on these incomplete digits has a test performance similar to one trained on complete data. On various continuous and binary data sets, we also show that MIWAE provides accurate single imputations, and is highly competitive with state-of-the-art methods.