Optimal adaptation for early stopping in statistical inverse problems

TL;DR

This paper develops optimal data-driven early stopping rules for iterative regularisation in linear inverse problems, achieving near-best estimation accuracy with theoretical guarantees and numerical validation.

Contribution

It introduces oracle adaptation bounds for residual-based stopping rules in spectral regularisation, advancing understanding of early stopping in statistical inverse problems.

Findings

Oracle bounds for residual-based stopping rules

Bias-variance transfer techniques for error analysis

Numerical illustration of adaptive early stopping

Abstract

For linear inverse problems $Y=\mathsf{A}\mu+\xi$, it is classical to recover the unknown signal $\mu$ by iterative regularisation methods $(\widehat \mu^{(m)}, m=0,1,\ldots)$ and halt at a data-dependent iteration $\tau$ using some stopping rule, typically based on a discrepancy principle, so that the weak (or prediction) squared-error $\|\mathsf{A}(\widehat \mu^{(\tau)}-\mu)\|^2$ is controlled. In the context of statistical estimation with stochastic noise $\xi$, we study oracle adaptation (that is, compared to the best possible stopping iteration) in strong squared-error $E[\|\hat \mu^{(\tau)}-\mu\|^2]$. For a residual-based stopping rule oracle adaptation bounds are established for general spectral regularisation methods. The proofs use bias and variance transfer techniques from weak prediction error to strong $L^2$-error, as well as convexity arguments and concentration bounds for the stochastic part. Adaptive early stopping for the Landweber method is studied in further detail and illustrated numerically.