A multilevel stochastic regularized first-order method with application to finite sum minimization

TL;DR

This paper introduces a multilevel stochastic regularized first-order method for nonconvex optimization that leverages hierarchical problem descriptions to improve efficiency, especially in data-fitting scenarios.

Contribution

It presents a novel stochastic multilevel framework that exploits hierarchical problem structures and provides convergence guarantees for nonconvex optimization.

Findings

Almost sure global convergence to a stationary point

Reduced computational cost in finite sum minimization

Effective in data-fitting problems with large datasets

Abstract

In this paper, we propose a multilevel stochastic framework for the solution of nonconvex unconstrained optimization problems. The proposed approach uses random regularized first-order models that exploit an available hierarchical description of the problem, being either in the classical variable space or in the function space, meaning that different levels of accuracy for the objective function are available. We propose a convergence analysis showing an almost sure global convergence of the method to a first order stationary point. The numerical behavior is tested on the solution of finite sum minimization problems. Differently from classical deterministic multilevel schemes, our stochastic method does not require the finest approximation to coincide with the original objective function along all the optimization process. This allows for significantly decreasing their cost, for instance in data-fitting problems, where considering all the data at each iteration can be avoided.