A variable dimension sketching strategy for nonlinear least-squares

TL;DR

This paper introduces a stochastic inexact Gauss-Newton method with variable subspace dimensions for nonlinear least-squares, reducing computational costs while maintaining convergence guarantees.

Contribution

It proposes a novel variable dimension sketching strategy for the Gauss-Newton method, with theoretical convergence analysis and practical effectiveness demonstrated through experiments.

Findings

Probabilistic bounds on iteration complexity.

Effective reduction in computational cost.

Successful numerical validation.

Abstract

We present a stochastic inexact Gauss-Newton method for the solution of nonlinear least-squares. To reduce the computational cost with respect to the classical method, at each iteration the proposed algorithm approximately minimizes the local model on a random subspace. The dimension of the subspace varies along the iterations, and two strategies are considered for its update: the first is based solely on the Armijo condition, the latter is based on information from the true Gauss-Newton model. Under suitable assumptions on the objective function and the random subspace, we prove a probabilistic bound on the number of iterations needed to drive the norm of the gradient below any given threshold. Moreover, we provide a theoretical analysis of the local behavior of the method. The numerical experiments demonstrate the effectiveness of the proposed method.