Optimal Unbiased Estimation for Expected Cumulative Cost

TL;DR

This paper introduces an unbiased Monte Carlo estimator for infinite-horizon discounted costs using optimal random truncation, providing explicit formulas and practical guidance for bias-variance tradeoff.

Contribution

It develops a novel unbiased estimator with explicit optimal truncation distributions, advancing Monte Carlo methods for stochastic process cost estimation.

Findings

Optimal random truncation distributions derived

Explicit bias-variance tradeoff analysis provided

Numerical experiments confirm theoretical advantages

Abstract

We consider estimating an expected infinite-horizon cumulative discounted cost/reward contingent on an underlying stochastic process by Monte Carlo simulation. An unbiased estimator based on truncating the cumulative cost at a random horizon is proposed. Explicit forms for the optimal distributions of the random horizon are given, and explicit expressions for the optimal random truncation level are obtained, leading to a full analysis of the bias-variance tradeoff when comparing this new class of randomized estimators with traditional fixed truncation estimators. Moreover, we characterize when the optimal randomized estimator is preferred over a fixed truncation estimator by considering the tradeoff between bias and variance. This comparison provides guidance on when to choose randomized estimators over fixed truncation estimators in practice. Numerical experiments substantiate the theoretical results.