Reduce the rank calculation of a high-dimensional sparse matrix based on network controllability theory

TL;DR

This paper introduces a fast, approximate method for estimating the rank of high-dimensional sparse matrices using network controllability theory and the cavity method, significantly reducing computation time compared to traditional SVD.

Contribution

The paper proposes a novel rank estimation algorithm based on maximum matching and the cavity method, inspired by network controllability theory, for high-dimensional sparse matrices.

Findings

The proposed method achieves high accuracy in rank estimation.

It significantly reduces computational time compared to SVD.

The approach is effective for large-scale sparse matrices.

Abstract

Numerical computing of the rank of a matrix is a fundamental problem in scientific computation. The datasets generated by the internet often correspond to the analysis of high-dimensional sparse matrices. Notwithstanding recent advances in the promotion of traditional singular value decomposition (SVD), an efficient estimation algorithm for the rank of a high-dimensional sparse matrix is still lacking. Inspired by the controllability theory of complex networks, we converted the rank of a matrix into maximum matching computing. Then, we established a fast rank estimation algorithm by using the cavity method, a powerful approximate technique for computing the maximum matching, to estimate the rank of a sparse matrix. In the merit of the natural low complexity of the cavity method, we showed that the rank of a high-dimensional sparse matrix can be estimated in a much faster way than SVD with high accuracy. Our method offers an efficient pathway to quickly estimate the rank of the high-dimensional sparse matrix when the time cost of computing the rank by SVD is unacceptable.