Worst-case low-rank approximations

TL;DR

This paper introduces wcPCA, a unified framework for worst-case low-rank approximation across heterogeneous domains, providing theoretical guarantees and demonstrating improved robustness in real-world applications.

Contribution

It develops a comprehensive framework for worst-case PCA, introduces new estimators like norm-minPCA and norm-maxregret, and extends the approach to matrix completion with theoretical optimality guarantees.

Findings

Improved worst-case performance in simulations and real-world data

Theoretical guarantees for estimators over source and target domains

Extensions to matrix completion with approximate worst-case optimality

Abstract

Real-world data in health, economics, and environmental sciences are often collected across heterogeneous domains (such as hospitals, regions, or time periods). In such settings, distributional shifts can make standard PCA unreliable, in that, for example, the leading principal components may explain substantially less variance in unseen domains than in the training domains. Existing approaches (such as FairPCA) have proposed to consider worst-case (rather than average) performance across multiple domains. This work develops a unified framework, called wcPCA, applies it to other objectives (resulting in the novel estimators such as norm-minPCA and norm-maxregret, which are better suited for applications with heterogeneous total variance) and analyzes their relationship. We prove that for all objectives, the estimators are worst-case optimal not only over the observed source domains but also over all target domains whose covariance lies in the convex hull of the (possibly normalized) source covariances. We establish consistency and asymptotic worst-case guarantees of empirical estimators. We extend our methodology to matrix completion, another problem that makes use of low-rank approximations, and prove approximate worst-case optimality for inductive matrix completion. Simulations and two real-world applications on ecosystem-atmosphere fluxes demonstrate marked improvements in worst-case performance, with only minor losses in average performance.