# When can we improve on sample average approximation for stochastic optimization?

## TL;DR

This paper compares sample average approximation with other methods like bagging, kernel smoothing, MLE, and Bayesian approaches in stochastic optimization, finding that SAA performs well especially with quadratic objectives, but can be outperformed by Bayesian methods.

## Contribution

The study provides a comparative analysis of SAA and alternative methods in stochastic optimization, highlighting conditions where SAA can be improved upon.

## Key findings

- SAA is highly effective for quadratic objectives.
- Bayesian approach outperforms SAA in certain scenarios.
- Bagging, MLE, and Bayesian methods perform well in portfolio optimization.

## Abstract

We explore the performance of sample average approximation in comparison with several other methods for stochastic optimization when there is information available on the underlying true probability distribution. The methods we evaluate are (a) bagging; (b) kernel smoothing; (c) maximum likelihood estimation (MLE); and (d) a Bayesian approach. We use two test sets, the first has a quadratic objective function allowing for very different types of interaction between the random component and the univariate decision variable. Here the sample average approximation is remarkably effective and only consistently outperformed by a Bayesian approach. The second test set is a portfolio optimization problem in which we use different covariance structures for a set of 5 stocks. Here bagging, MLE and a Bayesian approach all do well.