Data Sampling Strategies in Stochastic Algorithms for Empirical Risk Minimization

TL;DR

This paper develops and analyzes advanced data sampling strategies for stochastic gradient descent methods in big data optimization, introducing a flexible framework that broadens applicability and improves efficiency.

Contribution

It introduces new state-of-the-art sampling strategies for convex problems and a generalized framework applicable to diverse problems and sampling rules.

Findings

New sampling strategies outperform existing methods.

A flexible framework broadens the applicability of stochastic algorithms.

Enhanced efficiency in large-scale convex optimization.

Abstract

Gradient descent methods and especially their stochastic variants have become highly popular in the last decade due to their efficiency on big data optimization problems. In this thesis we present the development of data sampling strategies for these methods. In the first four chapters we focus on four views on the sampling for convex problems, developing and analyzing new state-of-the-art methods using non-standard data sampling strategies. Finally, in the last chapter we present a more flexible framework, which generalizes to more problems as well as more sampling rules.