On the rates of convergence of Parallelized Averaged Stochastic Gradient Algorithms

TL;DR

This paper analyzes the convergence rates of a parallelized averaged stochastic gradient algorithm, demonstrating its efficiency and asymptotic properties for high-dimensional data in distributed computing environments.

Contribution

It introduces a novel parallelized averaged stochastic gradient method with proven convergence rates and asymptotic normality for strongly convex objectives.

Findings

Established quadratic mean convergence rates.

Proved asymptotic normality of estimates.

Applicable to high-dimensional, distributed data settings.

Abstract

The growing interest for high dimensional and functional data analysis led in the last decade to an important research developing a consequent amount of techniques. Parallelized algorithms, which consist in distributing and treat the data into different machines, for example, are a good answer to deal with large samples taking values in high dimensional spaces. We introduce here a parallelized averaged stochastic gradient algorithm, which enables to treat efficiently and recursively the data, and so, without taking care if the distribution of the data into the machines is uniform. The rate of convergence in quadratic mean as well as the asymptotic normality of the parallelized estimates are given, for strongly and locally strongly convex objectives.