Worst-case convergence analysis of inexact gradient and Newton methods through semidefinite programming performance estimation

TL;DR

This paper introduces new semidefinite programming tools for analyzing the worst-case convergence of inexact gradient and Newton methods on convex functions, with applications to interior point methods.

Contribution

It extends performance estimation techniques to inexact methods and demonstrates their use in analyzing interior point algorithms' complexity.

Findings

Provides a novel performance estimation framework for inexact methods.

Derives worst-case complexity bounds for interior point methods.

Extends existing analysis tools to broader classes of optimization algorithms.

Abstract

We provide new tools for worst-case performance analysis of the gradient (or steepest descent) method of Cauchy for smooth strongly convex functions, and Newton's method for self-concordant functions, including the case of inexact search directions. The analysis uses semidefinite programming performance estimation, as pioneered by Drori and Teboulle [Mathematical Programming, 145(1-2):451-482, 2014], and extends recent performance estimation results for the method of Cauchy by the authors [Optimization Letters, 11(7), 1185-1199, 2017]. To illustrate the applicability of the tools, we demonstrate a novel complexity analysis of short step interior point methods using inexact search directions. As an example in this framework, we sketch how to give a rigorous worst-case complexity analysis of a recent interior point method by Abernethy and Hazan [PMLR, 48:2520-2528, 2016].