# Distributed Learning with Sparse Communications by Identification

## TL;DR

This paper introduces an asynchronous distributed optimization algorithm that reduces communication costs through random sparsification of updates, achieving linear convergence and enabling automatic dimension reduction.

## Contribution

It presents a novel asynchronous algorithm with sparse communication that converges linearly and identifies optimal sparse solutions for large-scale distributed learning.

## Key findings

- Algorithm converges linearly for strongly convex functions.
- Effectively identifies optimal sparse solutions.
- Enables automatic dimension reduction in distributed settings.

## Abstract

In distributed optimization for large-scale learning, a major performance limitation comes from the communications between the different entities. When computations are performed by workers on local data while a coordinator machine coordinates their updates to minimize a global loss, we present an asynchronous optimization algorithm that efficiently reduces the communications between the coordinator and workers. This reduction comes from a random sparsification of the local updates. We show that this algorithm converges linearly in the strongly convex case and also identifies optimal strongly sparse solutions. We further exploit this identification to propose an automatic dimension reduction, aptly sparsifying all exchanges between coordinator and workers.