Random-Subspace Sequential Quadratic Programming for Constrained Zeroth-Order Optimization

TL;DR

This paper introduces ZO-RS-SQP, a zeroth-order optimization method that efficiently solves high-dimensional constrained problems by combining random subspace sampling with sequential quadratic programming, achieving convergence with fewer evaluations.

Contribution

The paper proposes a novel zeroth-order SQP method that reduces evaluation costs by operating in low-dimensional subspaces, maintaining constraint handling and convergence guarantees.

Findings

Evaluation cost scales with subspace dimension, not ambient dimension.

Converges to first-order KKT points with high probability.

Numerical experiments demonstrate effectiveness on nonlinear constrained problems.

Abstract

We study nonlinear constrained optimization problems in which only function evaluations of the objective and constraints are available. Existing zeroth-order methods rely on noisy gradient and Jacobian surrogates in high dimensions, making it difficult to simultaneously achieve computational efficiency and accurate constraint satisfaction. We propose a zeroth-order random-subspace sequential quadratic programming method (ZO-RS-SQP) that combines two-point directional estimation with low-dimensional SQP updates. At each iteration, the method samples a random low-dimensional subspace, estimates the projected objective gradient and constraint Jacobians using two-point evaluations, and solves a reduced quadratic program to compute the step. As a result, the per-iteration evaluation cost scales with the subspace dimension rather than the ambient dimension, while retaining the structured linearized-constraint treatment of SQP. We also consider an Armijo line-search variant that improves robustness in practice. Under standard smoothness and regularity assumptions, we establish convergence to first-order KKT points with high probability. Numerical experiments illustrate the effectiveness of the proposed approach on nonlinear constrained problems.