Step size adaptation in first-order method for stochastic strongly convex programming

TL;DR

This paper introduces a simple, fast, and easily implementable first-order method for stochastic strongly convex optimization that achieves optimal convergence rates and outperforms existing methods in worst-case scenarios.

Contribution

The paper presents a novel first-order method for stochastic strongly convex optimization with an $O(1/n)$ convergence rate, outperforming peers in worst-case scenarios.

Findings

Achieves $O(1/n)$ convergence rate.

Simpler and easier to implement than existing methods.

Four times faster in worst-case scenarios.

Abstract

We propose a first-order method for stochastic strongly convex optimization that attains $O(1/n)$ rate of convergence, analysis show that the proposed method is simple, easily to implement, and in worst case, asymptotically four times faster than its peers. We derive this method from several intuitive observations that are generalized from existing first order optimization methods.