Geometric descent method for convex composite minimization

TL;DR

This paper introduces the geometric proximal gradient method (GeoPG), an extension of the geometric descent approach, which efficiently solves nonsmooth, strongly convex composite problems with optimal convergence rates.

Contribution

The paper develops GeoPG, a new algorithm that extends geometric descent to nonsmooth composite problems and proves its linear convergence with optimal rate.

Findings

GeoPG converges linearly with rate (1-1/√κ).

GeoPG outperforms Nesterov's method on ill-conditioned problems.

Numerical experiments validate GeoPG's efficiency.

Abstract

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrt{\kappa})$ and thus achieves the optimal rate among first-order methods, where $\kappa$ is the condition number of the problem. Numerical results on linear regression and logistic regression with elastic net regularization show that GeoPG compares favorably with Nesterov's accelerated proximal gradient method, especially when the problem is ill-conditioned.