Lifted Forward Planning in Relational Factored Markov Decision Processes with Concurrent Actions

TL;DR

This paper introduces Foreplan, a relational forward planner for relational factored MDPs with concurrent actions, which efficiently computes policies polynomially in the number of objects, significantly improving scalability.

Contribution

The paper presents Foreplan, the first efficient first-order relational planner for concurrent actions in factored MDPs, with polynomial complexity and an approximate version with error guarantees.

Findings

Foreplan achieves several orders of magnitude speedup over existing methods.

The approximate Foreplan maintains negligible error in empirical tests.

Theoretical analysis confirms polynomial complexity in the number of objects.

Abstract

When allowing concurrent actions in Markov Decision Processes, whose state and action spaces grow exponentially in the number of objects, computing a policy becomes highly inefficient, as it requires enumerating the joint of the two spaces. For the case of indistinguishable objects, we present a first-order representation to tackle the exponential blow-up in the action and state spaces. We propose Foreplan, an efficient relational forward planner, which uses the first-order representation allowing to compute policies in space and time polynomially in the number of objects. Thus, Foreplan significantly increases the number of planning problems solvable in an exact manner in reasonable time, which we underscore with a theoretical analysis. To speed up computations even further, we also introduce an approximate version of Foreplan, including guarantees on the error. Further, we provide an empirical evaluation of both Foreplan versions, demonstrating a speedup of several orders of magnitude. For the approximate version of Foreplan, we also empirically show that the induced error is often negligible.