Better Mini-Batch Algorithms via Accelerated Gradient Methods

TL;DR

This paper introduces an improved accelerated gradient algorithm for mini-batch stochastic convex optimization, demonstrating superior theoretical guarantees and practical performance over standard methods.

Contribution

It presents a novel accelerated gradient algorithm that overcomes limitations of standard methods, offering better guarantees and practical efficiency.

Findings

The new algorithm outperforms standard gradient methods in theory.

It achieves faster convergence in stochastic convex optimization.

Practical experiments confirm improved performance.

Abstract

Mini-batch algorithms have been proposed as a way to speed-up stochastic convex optimization problems. We study how such algorithms can be improved using accelerated gradient methods. We provide a novel analysis, which shows how standard gradient methods may sometimes be insufficient to obtain a significant speed-up and propose a novel accelerated gradient algorithm, which deals with this deficiency, enjoys a uniformly superior guarantee and works well in practice.