A Unified Convergence Rate Analysis of The Accelerated Smoothed Gap Reduction Algorithm

TL;DR

This paper provides a comprehensive convergence analysis of the ASGARD algorithm, covering various convexity settings and criteria, with optimal rates and new insights into dual and gap residuals, supported by numerical experiments.

Contribution

It introduces a unified convergence analysis framework for ASGARD that covers multiple convexity settings and criteria, with new convergence guarantees for gap and dual residuals.

Findings

Optimal convergence rates in all convexity settings.

New convergence guarantees for gap and dual residuals.

Numerical experiments validating the theoretical results.

Abstract

In this paper, we develop a unified convergence analysis framework for the Accelerated Smoothed GAp ReDuction algorithm (ASGARD) introduced in [20, Tran-Dinh et al, 2015] Unlike[20], the new analysis covers three settings in a single algorithm: general convexity, strong convexity, and strong convexity and smoothness. Moreover, we establish the convergence guarantees on three criteria: (i) gap function, (ii) primal objective residual, and (iii) dual objective residual. Our convergence rates are optimal (up to a constant factor) in all cases. While the convergence rate on the primal objective residual for the general convex case has been established in [20], we prove additional convergence rates on the gap function and the dual objective residual. The analysis for the last two cases is completely new. Our results provide a complete picture on the convergence guarantees of ASGARD. Finally, we present four different numerical experiments on a representative optimization model to verify our algorithm and compare it with Nesterov's smoothing technique.