Stochastic Auto-conditioned Fast Gradient Methods with Optimal Rates

TL;DR

This paper introduces a fully adaptive stochastic auto-conditioned fast gradient method that achieves optimal convergence rates without prior knowledge of problem parameters or line-search, addressing key limitations of existing methods.

Contribution

The authors propose a novel stochastic variant of the auto-conditioned fast gradient method that is adaptive to problem parameters and achieves optimal rates.

Findings

Achieves optimal iteration complexity of O(1/√ε).

Achieves optimal sample complexity of O(1/ε²).

Does not require line-search or prior parameter knowledge.

Abstract

Achieving optimal rates for stochastic composite convex optimization without prior knowledge of problem parameters remains a central challenge. In the deterministic setting, the auto-conditioned fast gradient method has recently been proposed to attain optimal accelerated rates without line-search procedures or prior knowledge of the Lipschitz smoothness constant, providing a natural prototype for parameter-free acceleration. However, extending this approach to the stochastic setting has proven technically challenging and remains open. Existing parameter-free stochastic methods either fail to achieve accelerated rates or rely on restrictive assumptions, such as bounded domains, bounded gradients, prior knowledge of the iteration horizon, or strictly sub-Gaussian noise. To address these limitations, we propose a stochastic variant of the auto-conditioned fast gradient method, referred to as stochastic AC-FGM. The proposed method is fully adaptive to the Lipschitz constant, the iteration horizon, and the noise level, enabling both adaptive stepsize selection and adaptive mini-batch sizing without line-search procedures. Under standard bounded conditional variance assumptions, we show that stochastic AC-FGM achieves the optimal iteration complexity of $O(1/\sqrt{\varepsilon})$ and the optimal sample complexity of $O(1/\varepsilon^2)$.