Dimension Reduction for Data with Heterogeneous Missingness

TL;DR

This paper introduces a bias-corrected Gram matrix to enable effective dimension reduction in high-dimensional data with heterogeneous missingness, improving the performance of various existing methods.

Contribution

It proposes a novel bias-corrected Gram matrix that addresses missingness issues, enhancing the robustness of dimension reduction techniques under heterogeneous missing data.

Findings

Bias-corrected Gram matrix improves dimension reduction accuracy

Method performs well on simulated and real datasets

Significantly enhances existing dimension reduction approaches

Abstract

Dimension reduction plays a pivotal role in analysing high-dimensional data. However, observations with missing values present serious difficulties in directly applying standard dimension reduction techniques. As a large number of dimension reduction approaches are based on the Gram matrix, we first investigate the effects of missingness on dimension reduction by studying the statistical properties of the Gram matrix with or without missingness, and then we present a bias-corrected Gram matrix with nice statistical properties under heterogeneous missingness. Extensive empirical results, on both simulated and publicly available real datasets, show that the proposed unbiased Gram matrix can significantly improve a broad spectrum of representative dimension reduction approaches.