Anisotropic Proximal Gradient

TL;DR

This paper introduces a novel anisotropic proximal gradient algorithm for nonconvex composite minimization, demonstrating improved convergence and practical performance over traditional Euclidean methods.

Contribution

It proposes a new anisotropic scheme based on dual space preconditioning, with theoretical convergence guarantees and applications to regularized LPs and logistic regression.

Findings

Proves the anisotropic descent property is closed under pointwise average.

Establishes linear convergence under anisotropic proximal gradient dominance.

Numerical experiments show significant improvements over Euclidean methods.

Abstract

This paper studies a novel algorithm for nonconvex composite minimization which can be interpreted in terms of dual space nonlinear preconditioning for the classical proximal gradient method. The proposed scheme can be applied to additive composite minimization problems whose smooth part exhibits an anisotropic descent inequality relative to a reference function. It is proved that the anisotropic descent property is closed under pointwise average if the Bregman distance generated by the conjugate reference function is jointly convex. More specifically, for the exponential reference function we prove its closedness under pointwise conic combinations. We analyze the method's asymptotic convergence and prove its linear convergence under an anisotropic proximal gradient dominance condition. Applications are discussed including exponentially regularized LPs and logistic regression with nonsmooth regularization. In numerical experiments we show significant improvements of the proposed method over its Euclidean counterparts.