Self-concordant inclusions: A unified framework for path-following generalized Newton-type algorithms

TL;DR

This paper introduces a unified Newton-type framework for self-concordant inclusions, enabling efficient solutions with local quadratic convergence and a new path-following scheme that improves iteration complexity for various convex optimization problems.

Contribution

The authors develop a generalized Newton framework for self-concordant inclusions, unifying multiple schemes and introducing a new inexact path-following method with optimal iteration complexity.

Findings

Proven local quadratic convergence of full-step and damped-step algorithms.

Developed a two-phase inexact path-following scheme with optimal complexity.

Validated the approach with numerical examples on convex problems.

Abstract

We study a class of monotone inclusions called "self-concordant inclusion" which covers three fundamental convex optimization formulations as special cases. We develop a new generalized Newton-type framework to solve this inclusion. Our framework subsumes three schemes: full-step, damped-step and path-following methods as specific instances, while allows one to use inexact computation to form generalized Newton directions. We prove a local quadratic convergence of both the full-step and damped-step algorithms. Then, we propose a new two-phase inexact path-following scheme for solving this monotone inclusion which possesses an $\mathcal{O}\left(\sqrt{\nu}\log(1/\varepsilon)\right)$-worst-case iteration-complexity to achieve an $\varepsilon$-solution, where $\nu$ is the barrier parameter and $\varepsilon$ is a desired accuracy. As byproducts, we customize our scheme to solve three convex problems: convex-concave saddle-point, nonsmooth constrained convex program, and nonsmooth convex program with linear constraints. We also provide three numerical examples to illustrate our theory and compare with existing methods.