On Parallel or Distributed Asynchronous Iterations with Unbounded Delays and Possible Out of Order Messages or Flexible Communication for Convex Optimization Problems and Machine Learning

TL;DR

This paper investigates asynchronous iterative algorithms in parallel and distributed settings, focusing on unbounded delays and out-of-order messages, and introduces new convergence results for convex optimization and machine learning applications.

Contribution

It introduces the macroiteration sequence concept for analyzing convergence and provides a new convergence proof for algorithms with flexible communication.

Findings

Convergence established for asynchronous algorithms with unbounded delays.

Analysis of out-of-order message handling in distributed optimization.

Survey of asynchronous iteration methods for convex problems.

Abstract

We describe several features of parallel or distributed asynchronous iterative algorithms such as unbounded delays, possible out of order messages or flexible communication. We concentrate on the concept of macroiteration sequence which was introduced in order to study the convergence or termination of asynchronous iterations. A survey of asynchronous iterations for convex optimization problems is also presented. Finally, a new result of convergence for parallel or distributed asynchronous iterative algorithms with flexible communication for convex optimization problems and machine learning is proposed.