Accelerated Algorithms for a Class of Optimization Problems with Constraints

TL;DR

This paper introduces an accelerated framework using High-Order Tuners for solving constrained optimization problems, achieving faster convergence and maintaining feasibility in equality-constrained cases, even for nonconvex problems.

Contribution

The paper develops a novel accelerated approach that reformulates constrained problems as unconstrained loss minimization, extending to nonconvex cases with guaranteed feasibility.

Findings

Achieves faster convergence than existing gradient methods.

Maintains feasibility in equality-constrained problems.

Extends applicability to nonconvex optimization.

Abstract

This paper presents a framework to solve constrained optimization problems in an accelerated manner based on High-Order Tuners (HT). Our approach is based on reformulating the original constrained problem as the unconstrained optimization of a loss function. We start with convex optimization problems and identify the conditions under which the loss function is convex. Building on the insight that the loss function could be convex even if the original optimization problem is not, we extend our approach to a class of nonconvex optimization problems. The use of a HT together with this approach enables us to achieve a convergence rate better than state-of-the-art gradient-based methods. Moreover, for equality-constrained optimization problems, the proposed method ensures that the state remains feasible throughout the evolution, regardless of the convexity of the original problem.