A Block Successive Upper Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization

TL;DR

This paper introduces BSUM-M, a new primal-dual algorithm for large-scale linearly constrained convex optimization, demonstrating convergence and linear rates under certain conditions, applicable to various practical fields.

Contribution

The paper proposes the BSUM-M algorithm, a novel block successive upper-bound minimization method of multipliers, for efficiently solving large-scale convex problems with linear constraints.

Findings

Converges to optimal solutions under regularity conditions.

Capable of linear convergence without strong convexity.

Effective for applications in signal processing, wireless networking, and smart grids.

Abstract

Consider the problem of minimizing the sum of a smooth convex function and a separable nonsmooth convex function subject to linear coupling constraints. Problems of this form arise in many contemporary applications including signal processing, wireless networking and smart grid provisioning. Motivated by the huge size of these applications, we propose a new class of first order primal-dual algorithms called the block successive upper-bound minimization method of multipliers (BSUM-M) to solve this family of problems. The BSUM-M updates the primal variable blocks successively by minimizing locally tight upper-bounds of the augmented Lagrangian of the original problem, followed by a gradient type update for the dual variable in closed form. We show that under certain regularity conditions, and when the primal block variables are updated in either a deterministic or a random fashion, the BSUM-M converges to the set of optimal solutions. Moreover, in the absence of linear constraints, we show that the BSUM-M, which reduces to the block successive upper-bound minimization (BSUM) method, is capable of linear convergence without strong convexity.