CoDGraD: A Code-based Distributed Gradient Descent Scheme for Decentralized Convex Optimization

TL;DR

This paper introduces CoDGraD, a novel distributed gradient descent algorithm that uses coding techniques to mitigate stragglers in decentralized convex optimization networks, ensuring reliable convergence.

Contribution

It proposes a new coded distributed gradient descent method for decentralized networks, improving robustness against stragglers and providing convergence analysis.

Findings

Effective mitigation of stragglers in decentralized networks.

Convergence guarantees for the proposed algorithm.

Numerical simulations demonstrating improved performance.

Abstract

In this paper, we consider a large network containing many regions such that each region is equipped with a worker with some data processing and communication capability. For such a network, some workers may become stragglers due to the failure or heavy delay on computing or communicating. To resolve the above straggling problem, a coded scheme that introduces certain redundancy for every worker was recently proposed, and a gradient coding paradigm was developed to solve convex optimization problems when the network has a centralized fusion center. In this paper, we propose an iterative distributed algorithm, referred as Code-Based Distributed Gradient Descent algorithm (CoDGraD), to solve convex optimization problems over distributed networks. In each iteration of the proposed algorithm, an active worker shares the coded local gradient and approximated solution of the convex optimization problem with non-straggling workers at the adjacent regions only. In this paper, we also provide the consensus and convergence analysis for the CoDGraD algorithm and we demonstrate its performance via numerical simulations.