Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlo

TL;DR

This paper introduces HH-VAEM, a hierarchical VAE with Hamiltonian Monte Carlo, to improve missing data imputation and acquisition, outperforming existing methods in accuracy and efficiency.

Contribution

The paper proposes a novel hierarchical VAE model with Hamiltonian Monte Carlo for better inference on mixed-type incomplete data, addressing limitations of previous single-layer Gaussian VAEs.

Findings

HH-VAEM outperforms existing baselines in missing data imputation.

The method improves supervised learning with missing features.

Sampling-based information gain computation is more effective than Gaussian-based alternatives.

Abstract

Variational Autoencoders (VAEs) have recently been highly successful at imputing and acquiring heterogeneous missing data. However, within this specific application domain, existing VAE methods are restricted by using only one layer of latent variables and strictly Gaussian posterior approximations. To address these limitations, we present HH-VAEM, a Hierarchical VAE model for mixed-type incomplete data that uses Hamiltonian Monte Carlo with automatic hyper-parameter tuning for improved approximate inference. Our experiments show that HH-VAEM outperforms existing baselines in the tasks of missing data imputation and supervised learning with missing features. Finally, we also present a sampling-based approach for efficiently computing the information gain when missing features are to be acquired with HH-VAEM. Our experiments show that this sampling-based approach is superior to alternatives based on Gaussian approximations.