Adaptive Smoothing Algorithms for Nonsmooth Composite Convex Minimization

TL;DR

This paper introduces an adaptive smoothing algorithm for nonsmooth convex optimization that combines Nesterov's technique with a homotopy strategy, achieving optimal iteration complexity and automatic parameter updates.

Contribution

The paper develops a new adaptive smoothing algorithm with optimal complexity, integrating a homotopy strategy and customizing it for various convex optimization problems.

Findings

Achieves $ ext{O}(1/\varepsilon)$ iteration complexity.

Automatically updates the smoothness parameter during iterations.

Demonstrates effectiveness through numerical experiments.

Abstract

We propose an adaptive smoothing algorithm based on Nesterov's smoothing technique in \cite{Nesterov2005c} for solving "fully" nonsmooth composite convex optimization problems. Our method combines both Nesterov's accelerated proximal gradient scheme and a new homotopy strategy for smoothness parameter. By an appropriate choice of smoothing functions, we develop a new algorithm that has the $\mathcal{O}\left(\frac{1}{\varepsilon}\right)$-worst-case iteration-complexity while preserves the same complexity-per-iteration as in Nesterov's method and allows one to automatically update the smoothness parameter at each iteration. Then, we customize our algorithm to solve four special cases that cover various applications. We also specify our algorithm to solve constrained convex optimization problems and show its convergence guarantee on a primal sequence of iterates. We demonstrate our algorithm through three numerical examples and compare it with other related algorithms.