Composite convex minimization involving self-concordant-like cost functions

TL;DR

This paper introduces a new optimization framework for convex functions with the self-concordant-like property, providing improved convergence guarantees and demonstrating practical effectiveness through numerical tests.

Contribution

It develops a variable metric method for minimizing sums of simple and self-concordant-like functions, with a novel step-size rule and enhanced convergence analysis.

Findings

The new algorithm outperforms traditional methods in convergence speed.

Numerical experiments confirm the theoretical improvements.

The approach effectively handles a broad class of convex functions with self-concordant-like properties.

Abstract

The self-concordant-like property of a smooth convex function is a new analytical structure that generalizes the self-concordant notion. While a wide variety of important applications feature the self-concordant-like property, this concept has heretofore remained unexploited in convex optimization. To this end, we develop a variable metric framework of minimizing the sum of a "simple" convex function and a self-concordant-like function. We introduce a new analytic step-size selection procedure and prove that the basic gradient algorithm has improved convergence guarantees as compared to "fast" algorithms that rely on the Lipschitz gradient property. Our numerical tests with real-data sets shows that the practice indeed follows the theory.