An Imputation-Consistency Algorithm for High-Dimensional Missing Data Problems and Beyond

TL;DR

This paper introduces a general imputation-consistency algorithm for high-dimensional missing data problems, addressing limitations of traditional methods like EM, and demonstrates its effectiveness in various complex models.

Contribution

It proposes a novel, general algorithm that iterates between imputation and consistency steps, applicable to high-dimensional data with missing values, under sparsity constraints.

Findings

Algorithm achieves consistency under broad conditions.

Effective in high-dimensional Gaussian graphical models.

Applicable to variable selection and random coefficient models.

Abstract

Missing data are frequently encountered in high-dimensional problems, but they are usually difficult to deal with using standard algorithms, such as the expectation-maximization (EM) algorithm and its variants. To tackle this difficulty, some problem-specific algorithms have been developed in the literature, but there still lacks a general algorithm. This work is to fill the gap: we propose a general algorithm for high-dimensional missing data problems. The proposed algorithm works by iterating between an imputation step and a consistency step. At the imputation step, the missing data are imputed conditional on the observed data and the current estimate of parameters; and at the consistency step, a consistent estimate is found for the minimizer of a Kullback-Leibler divergence defined on the pseudo-complete data. For high dimensional problems, the consistent estimate can be found under sparsity constraints. The consistency of the averaged estimate for the true parameter can be established under quite general conditions. The proposed algorithm is illustrated using high-dimensional Gaussian graphical models, high-dimensional variable selection, and a random coefficient model.