SOFAR: large-scale association network learning

TL;DR

The paper introduces SOFAR, a novel method for learning large-scale association networks that balances sparsity and orthogonality, with strong theoretical guarantees and broad applications in big data analysis.

Contribution

It proposes the SOFAR method using sparse singular value decomposition with orthogonality constraints, addressing the challenge of combining sparsity and orthogonality in network learning.

Findings

The SOFAR algorithm converges efficiently.

The method provides nonasymptotic error bounds.

Applications demonstrate improved network learning performance.

Abstract

Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsity and orthogonality have been two largely incompatible goals. To accommodate both features, in this paper we suggest the method of sparse orthogonal factor regression (SOFAR) via the sparse singular value decomposition with orthogonality constrained optimization to learn the underlying association networks, with broad applications to both unsupervised and supervised learning tasks such as biclustering with sparse singular value decomposition, sparse principal component analysis, sparse factor analysis, and spare vector autoregression analysis. Exploiting the framework of convexity-assisted nonconvex optimization, we derive nonasymptotic error bounds for the suggested procedure characterizing the theoretical advantages. The statistical guarantees are powered by an efficient SOFAR algorithm with convergence property. Both computational and theoretical advantages of our procedure are demonstrated with several simulation and real data examples.