Adaptive directional decomposition methods for nonconvex constrained optimization

TL;DR

This paper introduces adaptive directional decomposition algorithms for nonconvex constrained optimization, achieving improved iteration and oracle complexity bounds in deterministic and stochastic settings.

Contribution

The paper proposes a novel adaptive first-order framework with decomposition strategies and analyzes its complexity, advancing the theoretical understanding of nonconvex constrained optimization.

Findings

Achieves an iteration complexity of O(ε^{-2}) for deterministic problems.

Establishes high-probability oracle complexity bounds for stochastic problems.

Improves upon existing complexity bounds in the literature.

Abstract

In this paper, we study nonconvex constrained optimization problems with both equality and inequality constraints, covering deterministic and stochastic settings. We propose a novel first-order algorithm framework that employs a decomposition strategy to balance objective reduction and constraint satisfaction, together with adaptive update of stepsizes and merit parameters. Under certain conditions, the proposed adaptive directional decomposition methods attain an iteration complexity of order \(O(\epsilon^{-2})\) for finding an \(\epsilon\)-KKT point in the deterministic setting. In the stochastic setting, we further develop stochastic variants of approaches and analyze their theoretical properties by leveraging the perturbation theory. We establish the high-probability oracle complexity to find an $\epsilon$-KKT point of order \( \tilde O(\epsilon^{-4}, \epsilon^{-6}) \) (resp. \(\tilde O(\epsilon^{-3}, \epsilon^{-5}) \)) for gradient and constraint evaluations, in the absence (resp. presence) of sample-wise smoothness. To the best of our knowledge, the obtained complexity bounds are comparable to, or improve upon, the state-of-the-art results in the literature.