Online Pandora's Boxes and Bandits

TL;DR

This paper studies online decision-making in a generalized Pandora's box problem, proposing algorithms that approximate optimal offline policies under various constraints and linking it to multi-armed bandit problems.

Contribution

It introduces a reduction-based framework for online Pandora's box problems, extending classic models and providing approximation algorithms for complex constraints.

Findings

Pandora can achieve good approximation ratios in various scenarios.

The framework separates information acquisition from prize selection.

Extensions include multiple prizes and reinforcement learning settings.

Abstract

We consider online variations of the Pandora's box problem (Weitzman. 1979), a standard model for understanding issues related to the cost of acquiring information for decision-making. Our problem generalizes both the classic Pandora's box problem and the prophet inequality framework. Boxes are presented online, each with a random value and cost drew jointly from some known distribution. Pandora chooses online whether to open each box given its cost, and then chooses irrevocably whether to keep the revealed prize or pass on it. We aim for approximation algorithms against adversaries that can choose the largest prize over any opened box, and use optimal offline policies to decide which boxes to open (without knowledge of the value inside). We consider variations where Pandora can collect multiple prizes subject to feasibility constraints, such as cardinality, matroid, or knapsack constraints. We also consider variations related to classic multi-armed bandit problems from reinforcement learning. Our results use a reduction-based framework where we separate the issues of the cost of acquiring information from the online decision process of which prizes to keep. Our work shows that in many scenarios, Pandora can achieve a good approximation to the best possible performance.