A Confirmation of a Conjecture on the Feldman's Two-armed Bandit Problem

TL;DR

This paper proves a necessary and sufficient condition for the optimality of the myopic strategy in Feldman's two-armed bandit problem with general distributions, confirming a conjecture for Bernoulli cases and advancing understanding of bandit strategies.

Contribution

It provides a general criterion for myopic strategy optimality and confirms a specific conjecture for Bernoulli bandits, extending prior results.

Findings

Myopic strategy is optimal under certain conditions.

Confirmed conjecture that myopic strategy maximizes wins in Bernoulli bandits.

Established a necessary and sufficient condition for strategy optimality.

Abstract

Myopic strategy is one of the most important strategies when studying bandit problems. In this paper, we consider the two-armed bandit problem proposed by Feldman. With general distributions and utility functions, we obtain a necessary and sufficient condition for the optimality of the myopic strategy. As an application, we could solve Nouiehed and Ross's conjecture for Bernoulli two-armed bandit problems that myopic strategy stochastically maximizes the number of wins.