# Distributed Dual Coordinate Ascent in General Tree Networks and Communication Network Effect on Synchronous Machine Learning

## TL;DR

This paper analyzes the convergence rate of distributed dual coordinate ascent algorithms in general tree networks, considering communication delays, and optimizes the algorithm to improve convergence speed in large-scale distributed machine learning.

## Contribution

It generalizes distributed dual coordinate ascent to tree networks, provides convergence rate analysis, and optimizes the algorithm considering communication delays.

## Key findings

- Convergence rate can be recursively characterized in tree networks.
- Optimizing local iterations based on communication delays improves convergence.
- The algorithm is effective in scenarios with communication constraints.

## Abstract

Due to the big size of data and limited data storage volume of a single computer or a single server, data are often stored in a distributed manner. Thus, performing large-scale machine learning operations with the distributed datasets through communication networks is often required. In this paper, we study the convergence rate of the distributed dual coordinate ascent for distributed machine learning problems in a general tree-structured network. Since a tree network model can be understood as the generalization of a star network model, our algorithm can be thought of as the generalization of the distributed dual coordinate ascent in a star network model. We provide the convergence rate of the distributed dual coordinate ascent over a general tree network in a recursive manner and analyze the network effect on the convergence rate. Secondly, by considering network communication delays, we optimize the distributed dual coordinate ascent algorithm to maximize its convergence speed. From our analytical result, we can choose the optimal number of local iterations depending on the communication delay severity to achieve the fastest convergence speed. In numerical experiments, we consider machine learning scenarios over communication networks, where local workers cannot directly reach to a central node due to constraints in communication, and demonstrate that the usability of our distributed dual coordinate ascent algorithm in tree networks. Additionally, we show that adapting number of local and global iterations to network communication delays in the distributed dual coordinated ascent algorithm can improve its convergence speed.