A Bregman Splitting Algorithm for Distributed Optimization over Networks

TL;DR

This paper introduces a novel distributed optimization algorithm, D-FBBS, that efficiently handles asynchronous communication and stochastic networks, with proven convergence rates and superior performance in sensor fusion applications.

Contribution

The paper presents a new distributed Bregman splitting algorithm that works asynchronously and in stochastic networks, with established convergence guarantees and practical advantages.

Findings

Non-ergodic convergence rate of o(1/k) over fixed networks.

Ergodic convergence rate of O(1/k) over stochastic networks.

Superior performance in sensor fusion compared to existing methods.

Abstract

We consider distributed optimization problems in which a group of agents are to collaboratively seek the global optimum through peer-to-peer communication networks. The problem arises in various application areas, such as resource allocation, sensor fusion and distributed learning. We propose a general efficient distributed algorithm--termed Distributed Forward-Backward Bregman Splitting (D-FBBS)--to simultaneously solve the above primal problem as well as its dual based on Bregman method and operator splitting. The proposed algorithm allows agents to communicate asynchronously and thus lends itself to stochastic networks. This algorithm belongs to the family of general proximal point algorithms and is shown to have close connections with some existing well-known algorithms when dealing with fixed networks. However, we will show that it is generally different from the existing ones due to its effectiveness in handling stochastic networks. With proper assumptions, we establish a non-ergodic convergence rate of o(1/k) in terms of fixed point residuals over fixed networks both for D-FBBS and its inexact version (ID-FBBS) which is more computationally efficient and an ergodic convergence rate of O(1/k) for D-FBBS over stochastic networks respectively. We also apply the proposed algorithm to sensor fusion problems to show its superior performance compared to existing methods.