Gaussian One-Armed Bandit and Optimization of Batch Data Processing

TL;DR

This paper analyzes the minimax strategy for Gaussian one-armed bandits in batch data processing, deriving equations to compute optimal strategies and showing batch processing's near-optimality in large-sample scenarios.

Contribution

It introduces a recursive integro-difference and PDE framework for minimax strategies in Gaussian bandits with batch data, connecting game theory with numerical solutions.

Findings

Minimax risk can be approximated by PDE solutions for large batch numbers.

Batch data processing nearly matches optimal one-by-one processing in large-sample cases.

Theoretical equations enable numerical computation of strategies and risks.

Abstract

We consider the minimax setup for Gaussian one-armed bandit problem, i.e. the two-armed bandit problem with Gaussian distributions of incomes and known distribution corresponding to the first arm. This setup naturally arises when the optimization of batch data processing is considered and there are two alternative processing methods available with a priori known efficiency of the first method. One should estimate the efficiency of the second method and provide predominant usage of the most efficient of both them. According to the main theorem of the theory of games minimax strategy and minimax risk are searched for as Bayesian ones corresponding to the worst-case prior distribution. As a result, we obtain the recursive integro-difference equation and the second order partial differential equation in the limiting case as the number of batches goes to infinity. This makes it possible to determine minimax risk and minimax strategy by numerical methods. If the number of batches is large enough we show that batch data processing almost does not influence the control performance, i.e. the value of the minimax risk. Moreover, in case of Bernoulli incomes and large number of batches, batch data processing provides almost the same minimax risk as the optimal one-by-one data processing.