Mirror Descent and Convex Optimization Problems With Non-Smooth Inequality Constraints

TL;DR

This paper develops and unifies first-order Mirror Descent methods for convex optimization problems with non-smooth inequality constraints, adaptable to various problem settings without requiring Lipschitz constants.

Contribution

It introduces adaptive Mirror Descent algorithms for different convex optimization scenarios, including non-Lipschitz objectives, with a focus on practical parameter-free methods.

Findings

Methods work without prior knowledge of Lipschitz constants.

Algorithms handle both smooth and non-smooth objectives.

Addresses dual problem solution recovery.

Abstract

We consider the problem of minimization of a convex function on a simple set with convex non-smooth inequality constraint and describe first-order methods to solve such problems in different situations: smooth or non-smooth objective function; convex or strongly convex objective and constraint; deterministic or randomized information about the objective and constraint. We hope that it is convenient for a reader to have all the methods for different settings in one place. Described methods are based on Mirror Descent algorithm and switching subgradient scheme. One of our focus is to propose, for the listed different settings, a Mirror Descent with adaptive stepsizes and adaptive stopping rule. This means that neither stepsize nor stopping rule require to know the Lipschitz constant of the objective or constraint. We also construct Mirror Descent for problems with objective function, which is not Lipschitz continuous, e.g. is a quadratic function. Besides that, we address the problem of recovering the solution of the dual problem.