# ASYNC: A Cloud Engine with Asynchrony and History for Distributed Machine Learning

## TL;DR

ASYNC is a cloud framework designed to facilitate the implementation and experimentation of asynchronous and history-based optimization methods in distributed machine learning, addressing limitations of existing cloud engines.

## Contribution

It introduces a modular framework supporting asynchrony and history, enabling practical implementation of asynchronous optimization algorithms on distributed systems.

## Key findings

- Successfully implemented asynchronous SGD and SAGA in ASYNC
- Demonstrated ease of use for both synchronous and asynchronous variants
- Provides a flexible platform for distributed machine learning optimization

## Abstract

ASYNC is a framework that supports the implementation of asynchrony and history for optimization methods on distributed computing platforms. The popularity of asynchronous optimization methods has increased in distributed machine learning. However, their applicability and practical experimentation on distributed systems are limited because current bulk-processing cloud engines do not provide a robust support for asynchrony and history. With introducing three main modules and bookkeeping system-specific and application parameters, ASYNC provides practitioners with a framework to implement asynchronous machine learning methods. To demonstrate ease-of-implementation in ASYNC, the synchronous and asynchronous variants of two well-known optimization methods, stochastic gradient descent and SAGA, are demonstrated in ASYNC.