Early stopping for statistical inverse problems via truncated SVD estimation

TL;DR

This paper investigates data-driven early stopping rules for truncated SVD estimators in statistical inverse problems, analyzing their ability to achieve optimality with reduced computational cost.

Contribution

It provides a detailed analysis of sequential early stopping rules for truncated SVD, including lower and upper bounds, and proposes a hybrid approach for improved efficiency.

Findings

Optimal sequential adaptation is feasible under certain conditions.

Residual-based stopping rules can achieve statistical optimality.

A hybrid two-step approach reduces computational complexity while maintaining oracle inequalities.

Abstract

We consider truncated SVD (or spectral cut-off, projection) estimators for a prototypical statistical inverse problem in dimension $D$. Since calculating the singular value decomposition (SVD) only for the largest singular values is much less costly than the full SVD, our aim is to select a data-driven truncation level $\widehat m\in\{1,\ldots,D\}$ only based on the knowledge of the first $\widehat m$ singular values and vectors. We analyse in detail whether sequential {\it early stopping} rules of this type can preserve statistical optimality. Information-constrained lower bounds and matching upper bounds for a residual based stopping rule are provided, which give a clear picture in which situation optimal sequential adaptation is feasible. Finally, a hybrid two-step approach is proposed which allows for classical oracle inequalities while considerably reducing numerical complexity.