A Simple Convergence Analysis of Bregman Proximal Gradient Algorithm

TL;DR

This paper offers a simplified convergence analysis for the Bregman proximal gradient algorithm, demonstrating tighter bounds and improved performance through a novel perspective and line search variant.

Contribution

It introduces a new, simpler convergence analysis by linking the Bregman proximal gradient method to the proximal point algorithm, and proposes an effective backtracking line search variant.

Findings

Tighter convergence bounds than previous analyses.

Line search variant significantly improves convergence speed.

The analysis simplifies understanding of Bregman proximal methods.

Abstract

In this paper, we provide a simple convergence analysis of proximal gradient algorithm with Bregman distance, which provides a tighter bound than existing result. In particular, for the problem of minimizing a class of convex objective functions, we show that proximal gradient algorithm with Bregman distance can be viewed as proximal point algorithm that incorporates another Bregman distance. Consequently, the convergence result of the proximal gradient algorithm with Bregman distance follows directly from that of the proximal point algorithm with Bregman distance, and this leads to a simpler convergence analysis with a tighter convergence bound than existing ones. We further propose and analyze the backtracking line search variant of the proximal gradient algorithm with Bregman distance. Simulation results show that the line search method significantly improves the convergence performance of the algorithm.