Primal-Dual Incremental Gradient Method for Nonsmooth and Convex Optimization Problems

TL;DR

This paper introduces a primal-dual incremental gradient method for nonsmooth convex finite-sum problems with conic constraints, achieving improved convergence rates and practical efficiency over existing methods.

Contribution

The paper proposes a novel primal-dual incremental gradient scheme that reduces projection and gradient computation costs for constrained nonsmooth convex optimization.

Findings

Achieves asymptotic sublinear convergence rate

Outperforms existing incremental gradient methods in experiments

Handles conic constraints efficiently

Abstract

In this paper, we consider a nonsmooth convex finite-sum problem with a conic constraint. To overcome the challenge of projecting onto the constraint set and computing the full (sub)gradient, we introduce a primal-dual incremental gradient scheme where only a component function and two constraints are used to update each primal-dual sub-iteration in a cyclic order. We demonstrate an asymptotic sublinear rate of convergence in terms of suboptimality and infeasibility which is an improvement over the state-of-the-art incremental gradient schemes in this setting. Numerical results suggest that the proposed scheme compares well with competitive methods.