An Anytime Algorithm for Decision Making under Uncertainty

TL;DR

This paper introduces an anytime algorithm for decision-making under uncertainty that incrementally constructs policies, providing useful solutions quickly before reaching optimality, especially useful for large, complex problems.

Contribution

The paper presents a novel anytime algorithm that builds decision policies incrementally for influence diagrams, enabling early useful solutions in complex scenarios.

Findings

The algorithm produces valuable sub-optimal policies rapidly.

It converges to the optimal policy given sufficient computation time.

Effective on large decision problems with complex influence diagrams.

Abstract

We present an anytime algorithm which computes policies for decision problems represented as multi-stage influence diagrams. Our algorithm constructs policies incrementally, starting from a policy which makes no use of the available information. The incremental process constructs policies which includes more of the information available to the decision maker at each step. While the process converges to the optimal policy, our approach is designed for situations in which computing the optimal policy is infeasible. We provide examples of the process on several large decision problems, showing that, for these examples, the process constructs valuable (but sub-optimal) policies before the optimal policy would be available by traditional methods.