Nonlinear Laplacians: Tunable principal component analysis under directional prior information

TL;DR

This paper introduces nonlinear Laplacian-based spectral algorithms that leverage prior directional information to improve detection of rank-one signals in noisy data, outperforming traditional spectral methods.

Contribution

It proposes a novel class of algorithms using nonlinear Laplacians for signal detection, with methods to optimize the nonlinear function for enhanced performance.

Findings

Nonlinear Laplacian algorithms outperform standard spectral methods in detecting biased signals.

The optimal nonlinear function $\sigma$ can be tuned to minimize the required signal strength.

The approach is effective in models like sparse PCA and Gaussian planted submatrix problems.

Abstract

We introduce a new family of algorithms for detecting and estimating a rank-one signal from a noisy observation under prior information about that signal's direction, focusing on examples where the signal is known to have entries biased to be positive. Given a matrix observation $\mathbf{Y}$, our algorithms construct a nonlinear Laplacian, another matrix of the form $\mathbf{Y}+\mathrm{diag}(\sigma(\mathbf{Y1}))$ for a nonlinear $\sigma:\mathbb{R}\to\mathbb{R}$, and examine the top eigenvalue and eigenvector of this matrix. When $\mathbf{Y}$ is the (suitably normalized) adjacency matrix of a graph, our approach gives a class of algorithms that search for unusually dense subgraphs by computing a spectrum of the graph "deformed" by the degree profile $\mathbf{Y1}$. We study the performance of such algorithms compared to direct spectral algorithms (the case $\sigma=0$) on models of sparse principal component analysis with biased signals, including the Gaussian planted submatrix problem. For such models, we rigorously characterize the strength of rank-one signal, as a function of $\sigma$, required for an outlier eigenvalue to appear in the spectrum of a nonlinear Laplacian matrix. While identifying the $\sigma$ that minimizes the required signal strength in closed form seems intractable, we explore three approaches to design $\sigma$ numerically: exhaustively searching over simple classes of $\sigma$, learning $\sigma$ from datasets of problem instances, and tuning $\sigma$ using black-box optimization of the critical signal strength. We find both theoretically and empirically that, if $\sigma$ is chosen appropriately, then nonlinear Laplacian spectral algorithms substantially outperform direct spectral algorithms, while retaining the conceptual simplicity of spectral methods compared to broader classes of computations like approximate message passing or general first order methods.