Adaptive Policies for Sequential Sampling under Incomplete Information and a Cost Constraint

TL;DR

This paper develops adaptive policies for sequential sampling from multiple populations to maximize long-term average outcomes under cost constraints, ensuring convergence to optimal values even with unknown distributions.

Contribution

It introduces a class of consistent adaptive policies that guarantee convergence to the true optimal outcome under incomplete information and cost constraints.

Findings

Policies achieve almost sure convergence to the true mean outcomes.

Simulation shows different policies have varying convergence rates.

The approach effectively balances exploration and exploitation under cost limits.

Abstract

We consider the problem of sequential sampling from a finite number of independent statistical populations to maximize the expected infinite horizon average outcome per period, under a constraint that the expected average sampling cost does not exceed an upper bound. The outcome distributions are not known. We construct a class of consistent adaptive policies, under which the average outcome converges with probability 1 to the true value under complete information for all distributions with finite means. We also compare the rate of convergence for various policies in this class using simulation.