A single-phase, proximal path-following framework

TL;DR

This paper introduces a novel single-phase proximal path-following algorithm for constrained convex problems, effectively handling non-smooth objectives and improving convergence without initial phase requirements.

Contribution

It presents a new proximal, path-following framework that achieves optimal iteration complexity without the need for an initial phase, even with inexact proximal-Newton directions.

Findings

Achieves $ ext{O}(rac{1}{ ext{sqrt}( u)} ext{log}(1/ ext{epsilon}))$ iteration complexity.

Handles non-smooth objectives via proximal operators without dimension lifting.

Demonstrates effectiveness through three numerical examples.

Abstract

We propose a new proximal, path-following framework for a class of constrained convex problems. We consider settings where the nonlinear---and possibly non-smooth---objective part is endowed with a proximity operator, and the constraint set is equipped with a self-concordant barrier. Our approach relies on the following two main ideas. First, we re-parameterize the optimality condition as an auxiliary problem, such that a good initial point is available; by doing so, a family of alternative paths towards the optimum is generated. Second, we combine the proximal operator with path-following ideas to design a single-phase, proximal, path-following algorithm. Our method has several advantages. First, it allows handling non-smooth objectives via proximal operators; this avoids lifting the problem dimension in order to accommodate non-smooth components in optimization. Second, it consists of only a \emph{single phase}: While the overall convergence rate of classical path-following schemes for self-concordant objectives does not suffer from the initialization phase, proximal path-following schemes undergo slow convergence, in order to obtain a good starting point \cite{TranDinh2013e}. In this work, we show how to overcome this limitation in the proximal setting and prove that our scheme has the same $\mathcal{O}(\sqrt{\nu}\log(1/\varepsilon))$ worst-case iteration-complexity with standard approaches \cite{Nesterov2004,Nesterov1994} without requiring an initial phase, where $\nu$ is the barrier parameter and $\varepsilon$ is a desired accuracy. Finally, our framework allows errors in the calculation of proximal-Newton directions, without sacrificing the worst-case iteration complexity. We demonstrate the merits of our algorithm via three numerical examples, where proximal operators play a key role.