Trust-Region Stochastic Optimization with Variance Reduction Technique

TL;DR

The paper introduces TR-SVR, a trust-region stochastic optimization algorithm that uses variance reduction to enhance efficiency and stability for large-scale problems, combining SQP within a trust-region framework.

Contribution

It presents a novel trust-region method with variance reduction for stochastic optimization, integrating SQP for improved convergence and scalability.

Findings

Enhanced convergence stability with variance reduction

Improved computational efficiency for large-scale problems

Effective handling of stochastic objective functions

Abstract

We propose a novel algorithm, TR-SVR, for solving unconstrained stochastic optimization problems. This method builds on the trust-region framework, which effectively balances local and global exploration in optimization tasks. TR-SVR incorporates variance reduction techniques to improve both computational efficiency and stability when addressing stochastic objective functions. The algorithm applies a sequential quadratic programming (SQP) approach within the trust-region framework, solving each subproblem approximately using variance-reduced gradient estimators. This integration ensures a robust convergence mechanism while maintaining efficiency, making TR-SVR particularly suitable for large-scale stochastic optimization challenges.