On the Upper Bounds for the Matrix Spectral Norm

TL;DR

This paper introduces a new Counterbalance estimator for efficiently estimating upper bounds on a matrix's spectral norm using only matrix-vector products, outperforming traditional methods especially for matrices with fast-decaying spectra.

Contribution

The paper presents a novel Counterbalance estimator that provides tighter upper bounds on spectral norms with probabilistic guarantees, improving over standard approaches like the power method.

Findings

The estimator yields significantly tighter bounds in synthetic and real-world experiments.

It is particularly effective for matrices with fast-decaying spectra.

The method offers probabilistic guarantees on underestimation.

Abstract

We consider the problem of estimating the spectral norm of a matrix using only matrix-vector products. We propose a new Counterbalance estimator that provides upper bounds on the norm and derive probabilistic guarantees on its underestimation. Compared to standard approaches such as the power method, the proposed estimator produces significantly tighter upper bounds in both synthetic and real-world settings. Our method is especially effective for matrices with fast-decaying spectra, such as those arising in deep learning and inverse problems.