Accelerated forward-backward method with fast convergence rate for nonsmooth convex optimization beyond differentiability

TL;DR

This paper introduces a novel accelerated proximal gradient method, SAPG, that achieves fast convergence rates for nonsmooth convex optimization by combining smoothing techniques with extrapolation, and demonstrates its effectiveness through numerical experiments.

Contribution

The paper develops the SAPG algorithm that integrates smoothing with extrapolation to accelerate convergence in nonsmooth convex optimization, extending existing methods beyond differentiability.

Findings

Achieves a global convergence rate of o(ln^σ k / k) for objective values.

Proves convergence of the sequence to an optimal solution.

Demonstrates effectiveness and efficiency through numerical experiments.

Abstract

We propose an accelerated forward-backward method with fast convergence rate for finding a minimizer of a decomposable nonsmooth convex function over a closed convex set, and name it smoothing accelerated proximal gradient (SAPG) algorithm. The proposed algorithm combines the smoothing method with the proximal gradient algorithm with extrapolation $\frac{k-1}{k+\alpha-1}$ and $\alpha>3$. The updating rule of smoothing parameter $\mu_k$ is a smart scheme and guarantees the global convergence rate of $o(\ln^{\sigma}k/k)$ with $\sigma\in(\frac{1}{2},1]$ on the objective function values. Moreover, we prove that the sequence is convergent to an optimal solution of the problem. Furthermore, we introduce an error term in the SAPG algorithm to get the inexact smoothing accelerated proximal gradient algorithm. And we obtain the same convergence results as the SAPG algorithm under the summability condition on the errors. Finally, numerical experiments show the effectiveness and efficiency of the proposed algorithm.