Heuristic adaptive fast gradient method in stochastic optimization tasks

TL;DR

This paper introduces a heuristic adaptive fast gradient method for stochastic optimization, demonstrating improved practical convergence rates over existing methods, despite lacking complete theoretical guarantees.

Contribution

The paper proposes a new heuristic adaptive fast gradient method that empirically outperforms current optimization techniques in stochastic tasks.

Findings

Better practical convergence rates than popular methods

Justification of the heuristic approach with identified theoretical limitations

Potential for improved optimization performance in stochastic problems

Abstract

In this paper, we present a heuristic adaptive fast gradient method. We show that in practice our method has a better convergence rate than popular today optimization methods. Moreover, we justify our method and point out some problems that do not allow us to obtain theoretical estimates.