Stochastic Optimization and Data Science

TL;DR

This paper discusses stochastic optimization problems in data science, focusing on statistical and learning perspectives, and reviews offline and online solution approaches for expectation minimization.

Contribution

It provides a motivation for stochastic optimization from statistical viewpoints and summarizes main offline and online methods for solving these problems.

Findings

Highlights the statistical motivation for stochastic optimization

Describes Monte Carlo and Sample Average Approximation methods

Explains Stochastic Approximation techniques

Abstract

This paper aims to motivate stochastic optimization problems from a statistical perspective and a statistical learning perspective, where the goal is to maximize the log-likelihood or minimize the population risk. We briefly describe the two main approaches: offline (Monte Carlo / Sample Average Approximation) and online (Stochastic Approximation) approaches -- to solve the expectation minimization problems.