A Statistical View of Column Subset Selection

TL;DR

This paper unifies the computer science and statistical perspectives on column subset selection, showing their equivalence and providing methods for efficient, robust subset selection in high-dimensional data.

Contribution

It demonstrates the equivalence of CSS and principal variables, frames both as maximum likelihood estimation, and develops practical methods for high-dimensional, incomplete, or censored data.

Findings

CSS and principal variables are equivalent approaches.

Efficient CSS using only summary statistics is possible.

CSS methods are robust to missing and censored data.

Abstract

We consider the problem of selecting a small subset of representative variables from a large dataset. In the computer science literature, this dimensionality reduction problem is typically formalized as Column Subset Selection (CSS). Meanwhile, the typical statistical formalization is to find an information-maximizing set of Principal Variables. This paper shows that these two approaches are equivalent, and moreover, both can be viewed as maximum likelihood estimation within a certain semi-parametric model. Within this model, we establish suitable conditions under which the CSS estimate is consistent in high dimensions, specifically in the proportional asymptotic regime where the number of variables over the sample size converges to a constant. Using these connections, we show how to efficiently (1) perform CSS using only summary statistics from the original dataset; (2) perform CSS in the presence of missing and/or censored data; and (3) select the subset size for CSS in a hypothesis testing framework.