Parallel and Distributed Methods for Nonconvex Optimization-Part I: Theory

TL;DR

This paper introduces a versatile framework for nonconvex optimization that unifies and enhances existing algorithms, enabling distributed and parallel implementations for large-scale problems.

Contribution

It presents a general, flexible algorithmic framework for nonconvex constrained optimization, improving convergence and enabling distributed, parallel solutions.

Findings

Framework unifies multiple SCA-based algorithms

Ensures convergence to stationary solutions

Facilitates distributed and parallel implementations

Abstract

In this two-part paper, we propose a general algorithmic framework for the minimization of a nonconvex smooth function subject to nonconvex smooth constraints. The algorithm solves a sequence of (separable) strongly convex problems and mantains feasibility at each iteration. Convergence to a stationary solution of the original nonconvex optimization is established. Our framework is very general and flexible; it unifies several existing Successive Convex Approximation (SCA)-based algorithms such as (proximal) gradient or Newton type methods, block coordinate (parallel) descent schemes, difference of convex functions methods, and improves on their convergence properties. More importantly, and differently from current SCA approaches, it naturally leads to distributed and parallelizable implementations for a large class of nonconvex problems. This Part I is devoted to the description of the framework in its generality. In Part II we customize our general methods to several multi-agent optimization problems, mainly in communications and networking; the result is a new class of (distributed) algorithms that compare favorably to existing ad-hoc (centralized) schemes (when they exist).