A Parallel Stochastic Approximation Method for Nonconvex Multi-Agent Optimization Problems

TL;DR

This paper introduces a novel parallel stochastic approximation algorithm for nonconvex multi-agent optimization, enabling faster convergence and parallel updates in complex stochastic systems, with applications in wireless communication channels.

Contribution

It presents the first stochastic parallel Successive Convex Approximation framework for nonconvex stochastic sum-utility problems in multi-agent systems.

Findings

Faster convergence than existing stochastic gradient methods.

Achieves comparable or better sum-rates in wireless channel optimization.

Supports parallel updates for all users in multi-agent settings.

Abstract

Consider the problem of minimizing the expected value of a (possibly nonconvex) cost function parameterized by a random (vector) variable, when the expectation cannot be computed accurately (e.g., because the statistics of the random variables are unknown and/or the computational complexity is prohibitive). Classical sample stochastic gradient methods for solving this problem may empirically suffer from slow convergence. In this paper, we propose for the first time a stochastic parallel Successive Convex Approximation-based (best-response) algorithmic framework for general nonconvex stochastic sum-utility optimization problems, which arise naturally in the design of multi-agent systems. The proposed novel decomposition enables all users to update their optimization variables in parallel by solving a sequence of strongly convex subproblems, one for each user. Almost surely convergence to stationary points is proved. We then customize our algorithmic framework to solve the stochastic sum rate maximization problem over Single-Input-Single-Output (SISO) frequency-selective interference channels, multiple-input-multiple-output (MIMO) interference channels, and MIMO multiple-access channels. Numerical results show that our algorithms are much faster than state-of-the-art stochastic gradient schemes while achieving the same (or better) sum-rates.