Fast and Accurate Pseudoinverse with Sparse Matrix Reordering and Incremental Approach

TL;DR

FastPI is an incremental SVD-based method that efficiently computes the pseudoinverse of large, sparse matrices, enabling faster and accurate solutions for large-scale machine learning problems.

Contribution

The paper introduces FastPI, a novel incremental pseudoinverse algorithm that leverages sparse matrix reordering and division for improved efficiency and accuracy.

Findings

FastPI outperforms existing approximate methods in speed.

FastPI maintains high accuracy comparable to exact methods.

FastPI uses less memory than full SVD approaches.

Abstract

How can we compute the pseudoinverse of a sparse feature matrix efficiently and accurately for solving optimization problems? A pseudoinverse is a generalization of a matrix inverse, which has been extensively utilized as a fundamental building block for solving linear systems in machine learning. However, an approximate computation, let alone an exact computation, of pseudoinverse is very time-consuming due to its demanding time complexity, which limits it from being applied to large data. In this paper, we propose FastPI (Fast PseudoInverse), a novel incremental singular value decomposition (SVD) based pseudoinverse method for sparse matrices. Based on the observation that many real-world feature matrices are sparse and highly skewed, FastPI reorders and divides the feature matrix and incrementally computes low-rank SVD from the divided components. To show the efficacy of proposed FastPI, we apply them in real-world multi-label linear regression problems. Through extensive experiments, we demonstrate that FastPI computes the pseudoinverse faster than other approximate methods without loss of accuracy. %and uses much less memory compared to full-rank SVD based approach. Results imply that our method efficiently computes the low-rank pseudoinverse of a large and sparse matrix that other existing methods cannot handle with limited time and space.