Flexible Policy Construction by Information Refinement

TL;DR

This paper introduces a flexible, incremental algorithm for constructing decision functions in influence diagrams, which converges to optimality and improves expected utility with manageable computational costs.

Contribution

The paper presents a novel incremental tree construction algorithm for influence diagrams that approaches optimal decision functions with better computational efficiency.

Findings

Expected utility increases with tree size and Bayesian network calculations.

Algorithm converges to optimal decision functions asymptotically.

Performance is only a constant factor worse than dynamic programming.

Abstract

We report on work towards flexible algorithms for solving decision problems represented as influence diagrams. An algorithm is given to construct a tree structure for each decision node in an influence diagram. Each tree represents a decision function and is constructed incrementally. The improvements to the tree converge to the optimal decision function (neglecting computational costs) and the asymptotic behaviour is only a constant factor worse than dynamic programming techniques, counting the number of Bayesian network queries. Empirical results show how expected utility increases with the size of the tree and the number of Bayesian net calculations.