On the Convergence Analysis of an Inexact Preconditioned Stochastic Model-Based Algorithm

TL;DR

This paper analyzes an inexact stochastic model-based optimization algorithm with preconditioning, providing convergence guarantees under mild conditions and demonstrating its effectiveness through numerical experiments.

Contribution

It extends existing stochastic model-based algorithms to include preconditioning and inexactness, with new convergence analysis under mild assumptions.

Findings

Convergence guarantees under mild assumptions for weakly convex and convex problems.

Nonasymptotic and asymptotic convergence rates derived using the Moreau envelope.

Numerical experiments validate the theoretical convergence results.

Abstract

This paper focuses on investigating an inexact stochastic model-based optimization algorithm that integrates preconditioning techniques for solving stochastic composite optimization problems. The proposed framework unifies and extends the fixed-metric stochastic model-based algorithm to its preconditioned and inexact variants. Convergence guarantees are established under mild assumptions for both weakly convex and convex settings, without requiring smoothness or global Lipschitz continuity of the objective function. By assuming a local Lipschitz condition, we derive nonasymptotic and asymptotic convergence rates measured by the gradient of the Moreau envelope. Furthermore, convergence rates in terms of the distance to the optimal solution set are obtained under an additional quadratic growth condition on the objective function. Numerical experiment results demonstrate the theoretical findings for the proposed algorithm.