# Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies

## TL;DR

This paper introduces a new convergence analysis framework for stochastic L-BFGS, demonstrating improved theoretical rates and practical acceleration strategies that significantly enhance empirical performance on large-scale problems.

## Contribution

The paper provides a novel convergence analysis framework for stochastic L-BFGS and proposes practical acceleration strategies with theoretical validation.

## Key findings

- Improved convergence rates and computational complexities for stochastic L-BFGS.
- Practical acceleration strategies significantly speed up empirical performance.
- Experimental results show superior performance on large-scale logistic and ridge regression tasks.

## Abstract

We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. In addition, we propose several practical acceleration strategies to speed up the empirical performance of such algorithms. We also provide theoretical analyses for most of the strategies. Experiments on large-scale logistic and ridge regression problems demonstrate that our proposed strategies yield significant improvements vis-\`a-vis competing state-of-the-art algorithms.