Distributed Big-Data Optimization via Block-Iterative Convexification and Averaging

TL;DR

This paper introduces a distributed optimization algorithm for large-scale nonconvex problems in multi-agent networks, combining block-iterative convexification, gradient tracking, and consensus techniques to improve efficiency and convergence.

Contribution

It proposes a novel distributed method that optimizes subsets of variables iteratively, reducing computation and communication burdens in big-data nonconvex optimization.

Findings

Algorithm converges to stationary solutions.

Block size affects communication overhead and convergence speed.

Numerical results validate effectiveness of the proposed approach.

Abstract

In this paper, we study distributed big-data nonconvex optimization in multi-agent networks. We consider the (constrained) minimization of the sum of a smooth (possibly) nonconvex function, i.e., the agents' sum-utility, plus a convex (possibly) nonsmooth regularizer. Our interest is in big-data problems wherein there is a large number of variables to optimize. If treated by means of standard distributed optimization algorithms, these large-scale problems may be intractable, due to the prohibitive local computation and communication burden at each node. We propose a novel distributed solution method whereby at each iteration agents optimize and then communicate (in an uncoordinated fashion) only a subset of their decision variables. To deal with non-convexity of the cost function, the novel scheme hinges on Successive Convex Approximation (SCA) techniques coupled with i) a tracking mechanism instrumental to locally estimate gradient averages; and ii) a novel block-wise consensus-based protocol to perform local block-averaging operations and gradient tacking. Asymptotic convergence to stationary solutions of the nonconvex problem is established. Finally, numerical results show the effectiveness of the proposed algorithm and highlight how the block dimension impacts on the communication overhead and practical convergence speed.