Combinatorial Selection with Costly Information

TL;DR

This paper develops a framework for approximate optimization of costly information acquisition in stochastic combinatorial problems modeled as acyclic MDPs, enabling near-optimal solutions for various complex bandit superprocesses.

Contribution

It introduces a novel cost amortization and local approximation framework for arbitrary acyclic MDPs with matroid constraints, extending solutions to new and existing bandit superprocess variants.

Findings

Established bounds on optimal costs via cost amortization.

Achieved approximate solutions for Pandora's Box variants and the new Weighing Scale problem.

Provided new approximation algorithms for combinatorial problems with costly information.

Abstract

We consider a class of optimization problems over stochastic variables where the algorithm can learn information about the value of any variable through a series of costly steps; we model this information acquisition process as a Markov Decision Process (MDP). The algorithm's goal is to minimize the cost of its solution plus the cost of information acquisition, or alternately, maximize the value of its solution minus the cost of information acquisition. Such bandit superprocesses have been studied previously but solutions are known only for fairly restrictive special cases. We develop a framework for approximate optimization of bandit superprocesses that applies to arbitrary acyclic MDPs with a matroid feasibility constraint. Our framework establishes a bound on the optimal cost through a novel cost amortization; it then couples this bound with a notion of local approximation that allows approximate solutions for each component MDP in the superprocess to be composed without loss into a global approximation. We use this framework to obtain approximately optimal solutions for several variants of bandit superprocesses for both maximization and minimization. We obtain new approximations for combinatorial versions of the previously studied Pandora's Box with Optional Inspection and Pandora's Box with Partial Inspection; the less-studied Additive Pandora's Box problem; as well as a new problem that we call the Weighing Scale problem.