A Trust-Region Interior-Point Stochastic Sequential Quadratic Programming Method

TL;DR

This paper introduces a novel stochastic optimization algorithm combining trust-region, interior-point, and sequential quadratic programming techniques, designed for problems with stochastic objectives and deterministic constraints, with proven convergence and practical testing.

Contribution

It develops a new TR-IP-SSQP method that handles stochastic objectives with adaptive accuracy, integrating interior-point strategies for constraints, and proves its convergence under standard assumptions.

Findings

Demonstrates global almost-sure convergence to stationary points.

Shows practical effectiveness on CUTEst and logistic regression problems.

Provides a new framework for stochastic constrained optimization.

Abstract

In this paper, we propose a trust-region interior-point stochastic sequential quadratic programming (TR-IP-SSQP) method for solving optimization problems with a stochastic objective and deterministic nonlinear equality and inequality constraints. In this setting, exact evaluations of the objective function and its gradient are unavailable, but their stochastic estimates can be constructed. In particular, at each iteration our method builds stochastic oracles, which estimate the objective value and gradient to satisfy proper adaptive accuracy conditions with a fixed probability. To handle inequality constraints, we adopt an interior-point method (IPM), in which the barrier parameter follows a prescribed decaying sequence. Under standard assumptions, we establish global almost-sure convergence of the proposed method to first-order stationary points. We implement the method on a subset of problems from the CUTEst test set, as well as on logistic regression problems, to demonstrate its practical performance.