Almost-Optimal Local-Search Methods for Sparse Tensor PCA

TL;DR

This paper introduces new local-search algorithms for sparse tensor PCA that close the performance gap with the best polynomial-time methods, supported by rigorous theoretical analysis.

Contribution

It proposes novel local-search algorithms, including random-threshold variants, that outperform existing Markov chain methods in sparse tensor PCA.

Findings

Algorithms match the performance of the best polynomial-time procedures.

Random-threshold variants enable precise trajectory analysis.

Methods outperform prior local Markov chain approaches in multiple regimes.

Abstract

Local-search methods are widely employed in statistical applications, yet interestingly, their theoretical foundations remain rather underexplored, compared to other classes of estimators such as low-degree polynomials and spectral methods. Of note, among the few existing results recent studies have revealed a significant "local-computational" gap in the context of a well-studied sparse tensor principal component analysis (PCA), where a broad class of local Markov chain methods exhibits a notable underperformance relative to other polynomial-time algorithms. In this work, we propose a series of local-search methods that provably "close" this gap to the best known polynomial-time procedures in multiple regimes of the model, including and going beyond the previously studied regimes in which the broad family of local Markov chain methods underperforms. Our framework includes: (1) standard greedy and randomized greedy algorithms applied to the (regularized) posterior of the model; and (2) novel random-threshold variants, in which the randomized greedy algorithm accepts a proposed transition if and only if the corresponding change in the Hamiltonian exceeds a random Gaussian threshold-rather that if and only if it is positive, as is customary. The introduction of the random thresholds enables a tight mathematical analysis of the randomized greedy algorithm's trajectory by crucially breaking the dependencies between the iterations, and could be of independent interest to the community.