Data-Driven Estimation of Conditional Expectations, Application to Optimal Stopping and Reinforcement Learning

TL;DR

This paper introduces a data-driven method for estimating conditional expectations directly from training data, applicable to stochastic optimization problems like optimal stopping and reinforcement learning, without requiring known conditional densities.

Contribution

The work presents a novel, simple approach for estimating conditional expectations solely from data, extending to nonlinear systems in stochastic optimization.

Findings

Effective in estimating conditional expectations from data

Applicable to optimal stopping and reinforcement learning tasks

Demonstrates promising results in stochastic optimization problems

Abstract

When the underlying conditional density is known, conditional expectations can be computed analytically or numerically. When, however, such knowledge is not available and instead we are given a collection of training data, the goal of this work is to propose simple and purely data-driven means for estimating directly the desired conditional expectation. Because conditional expectations appear in the description of a number of stochastic optimization problems with the corresponding optimal solution satisfying a system of nonlinear equations, we extend our data-driven method to cover such cases as well. We test our methodology by applying it to Optimal Stopping and Optimal Action Policy in Reinforcement Learning.