Combinatorial Markov Search

TL;DR

This paper introduces a model called Combinatorial Markov Search, analyzing a complex decision problem involving sequential exploration of alternatives with rewards and costs, providing approximation algorithms with strong theoretical guarantees.

Contribution

It develops a novel framework for combinatorial search modeled as Markov Decision Processes and proves prophet inequalities with approximation guarantees under matroid constraints.

Findings

Provides a $rac{1}{2}- ext{epsilon}$ prophet inequality for the problem.

Establishes incentive-compatible mechanisms with constant Price of Anarchy.

Shows NP-hardness in simple cases generalizing Pandora's Box.

Abstract

A decisionmaker faces $n$ alternatives, each of which represents a potential reward. After investing costly resources into investigating the alternatives, the decisionmaker may select one, or more generally a feasible subset, and obtain the associated reward(s). The objective is to maximize the sum of rewards minus total costs invested. We consider this problem under a general model of an alternative as a "Markov Search Process," a type of undiscounted Markov Decision Process on a finite acyclic graph. Even simple cases generalize NP-hard problems such as Pandora's Box with nonobligatory inspection. Despite the apparently adaptive and interactive nature of the problem, we prove optimal prophet inequalities for this problem under a variety of combinatorial constraints. That is, we give approximation algorithms that interact with the alternatives sequentially, where each must be fully explored and either selected or else discarded before the next arrives. In particular, we obtain a computationally efficient $\frac{1}{2}-\epsilon$ prophet inequality for Combinatorial Markov Search subject to any matroid constraint. This result implies incentive-compatible mechanisms with constant Price of Anarchy for serving single-parameter agents when the agents strategically conduct independent, costly search processes to discover their values.