Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environment

TL;DR

This paper demonstrates that dynamic programming can be adapted to partially observed nautical environments, enabling safer navigation policies that outperform traditional methods and reduce measurement costs through controlled sensing.

Contribution

It introduces a novel approach to applying dynamic programming to POMDPs in nautical navigation, incorporating uncertainty and controlled sensing to improve safety and efficiency.

Findings

Navigation policies maintain safety under uncertainty

Policies outperform traditional MDP-based methods

Controlled sensing reduces measurement costs

Abstract

Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.