Fast Value Iteration for Goal-Directed Markov Decision Processes

TL;DR

This paper introduces techniques that leverage goal-directedness in Markov decision processes to significantly speed up value iteration, enhancing planning efficiency in non-deterministic environments.

Contribution

The paper presents novel methods specifically designed to exploit goal-directedness for accelerating value iteration in MDPs, demonstrating substantial empirical speedups.

Findings

Techniques achieve significant speedups in value iteration.

Empirical results confirm efficiency improvements.

Goal-directed exploitation enhances planning in non-deterministic settings.

Abstract

Planning problems where effects of actions are non-deterministic can be modeled as Markov decision processes. Planning problems are usually goal-directed. This paper proposes several techniques for exploiting the goal-directedness to accelerate value iteration, a standard algorithm for solving Markov decision processes. Empirical studies have shown that the techniques can bring about significant speedups.