Adaptive fast gradient method in stochastic optimization tasks

TL;DR

This paper introduces a novel stochastic adaptive fast gradient descent method utilizing mirror variants, with proven probabilistic guarantees, marking the first use of adaptivity in such stochastic optimization techniques.

Contribution

It presents the first adaptive stochastic gradient method based on mirror variants, with theoretical probabilistic deviation bounds.

Findings

First adaptive stochastic gradient method with mirror variants

Probabilistic bounds on large deviations proved

Demonstrates effectiveness in stochastic optimization tasks

Abstract

In this paper, we describe a stochastic adaptive fast gradient descent method based on the mirror variant of similar triangles method. To our knowledge, this is the first attempt to use adaptivity in stochastic method. Additionally, a main result was proved in terms of probabilities of large deviations.