Analysis of Truncated Orthogonal Iteration for Sparse Eigenvector Problems

TL;DR

This paper introduces two variants of Truncated Orthogonal Iteration for efficiently estimating multiple sparse eigenvectors in high-dimensional systems, with proven convergence and successful application to diverse datasets.

Contribution

The paper proposes novel algorithms for sparse eigenvector estimation, providing convergence analysis and demonstrating superior performance on real-world data.

Findings

Algorithms achieve state-of-the-art results quickly.

Methods work with minimal parameter tuning.

Effective on datasets like MNIST, temperature, and newsgroups.

Abstract

A wide range of problems in computational science and engineering require estimation of sparse eigenvectors for high dimensional systems. Here, we propose two variants of the Truncated Orthogonal Iteration to compute multiple leading eigenvectors with sparsity constraints simultaneously. We establish numerical convergence results for the proposed algorithms using a perturbation framework, and extend our analysis to other existing alternatives for sparse eigenvector estimation. We then apply our algorithms to solve the sparse principle component analysis problem for a wide range of test datasets, from simple simulations to real-world datasets including MNIST, sea surface temperature and 20 newsgroups. In all these cases, we show that the new methods get state of the art results quickly and with minimal parameter tuning.