Nesterov Acceleration for Equality-Constrained Convex Optimization via Continuously Differentiable Penalty Functions

TL;DR

This paper introduces a novel framework that reformulates constrained convex problems into unconstrained ones using a differentiable penalty function, enabling the application of Nesterov's accelerated method for improved convergence rates.

Contribution

It develops a new approach combining penalty functions and Nesterov acceleration for constrained convex optimization, with conditions ensuring convexity of the reformulated problem.

Findings

Achieves better convergence rates than existing first-order methods.

Provides sufficient conditions for convexity of the penalty function.

Demonstrates effectiveness through simulations.

Abstract

We propose a framework to use Nesterov's accelerated method for constrained convex optimization problems. Our approach consists of first reformulating the original problem as an unconstrained optimization problem using a continuously differentiable exact penalty function. This reformulation is based on replacing the Lagrange multipliers in the augmented Lagrangian of the original problem by Lagrange multiplier functions. The expressions of these Lagrange multiplier functions, which depend upon the gradients of the objective function and the constraints, can make the unconstrained penalty function non-convex in general even if the original problem is convex. We establish sufficient conditions on the objective function and the constraints of the original problem under which the unconstrained penalty function is convex. This enables us to use Nesterov's accelerated gradient method for unconstrained convex optimization and achieve a guaranteed rate of convergence which is better than the state-of-the-art first-order algorithms for constrained convex optimization. Simulations illustrate our results.