Stochastic Search with an Observable State Variable

TL;DR

This paper introduces nonparametric density estimation techniques to solve convex stochastic search problems with an observable state variable, providing new algorithms that adapt to problem characteristics and outperform traditional methods in certain cases.

Contribution

It proposes two novel solution methods using nonparametric density estimation for stochastic search problems influenced by an exogenous state variable, filling a gap in existing algorithms.

Findings

Dirichlet process weights can outperform kernel-based weights in some scenarios.

Nonparametric methods yield effective solutions for complex stochastic search problems.

The approaches are validated on synthetic and real-world inspired problems.

Abstract

In this paper we study convex stochastic search problems where a noisy objective function value is observed after a decision is made. There are many stochastic search problems whose behavior depends on an exogenous state variable which affects the shape of the objective function. Currently, there is no general purpose algorithm to solve this class of problems. We use nonparametric density estimation to take observations from the joint state-outcome distribution and use them to infer the optimal decision for a given query state. We propose two solution methods that depend on the problem characteristics: function-based and gradient-based optimization. We examine two weighting schemes, kernel-based weights and Dirichlet process-based weights, for use with the solution methods. The weights and solution methods are tested on a synthetic multi-product newsvendor problem and the hour-ahead wind commitment problem. Our results show that in some cases Dirichlet process weights offer substantial benefits over kernel based weights and more generally that nonparametric estimation methods provide good solutions to otherwise intractable problems.