Planning under periodic observations: bounds and bounding-based solutions

TL;DR

This paper addresses planning for robots with intermittent information updates, proposing bounds and bounding-based methods to efficiently solve complex decision problems with practical applications like planetary exploration.

Contribution

It introduces a new subclass of planning problems with periodic information updates and develops systematic performance bounds for these challenging Markov Decision Processes.

Findings

Performance bounds improve solution quality

Bounding-based methods are effective empirically

Time until information is revealed is a key factor

Abstract

We study planning problems faced by robots operating in uncertain environments with incomplete knowledge of state, and actions that are noisy and/or imprecise. This paper identifies a new problem sub-class that models settings in which information is revealed only intermittently through some exogenous process that provides state information periodically. Several practical domains fit this model, including the specific scenario that motivates our research: autonomous navigation of a planetary exploration rover augmented by remote imaging. With an eye to efficient specialized solution methods, we examine the structure of instances of this sub-class. They lead to Markov Decision Processes with exponentially large action-spaces but for which, as those actions comprise sequences of more atomic elements, one may establish performance bounds by comparing policies under different information assumptions. This provides a way in which to construct performance bounds systematically. Such bounds are useful because, in conjunction with the insights they confer, they can be employed in bounding-based methods to obtain high-quality solutions efficiently; the empirical results we present demonstrate their effectiveness for the considered problems. The foregoing has also alluded to the distinctive role that time plays for these problems -- more specifically: time until information is revealed -- and we uncover and discuss several interesting subtleties in this regard.