A Finite-State Controller Based Offline Solver for Deterministic POMDPs

TL;DR

This paper introduces DetMCVI, a Monte Carlo-based algorithm that constructs finite-state controllers to efficiently solve large deterministic POMDPs, demonstrated through simulations and real-world robot mapping.

Contribution

The paper presents DetMCVI, a novel offline solver for DetPOMDPs that builds finite-state controllers and outperforms existing methods in large problem settings.

Findings

DetMCVI successfully solves large DetPOMDPs with high success rates.

The algorithm outperforms existing baselines in benchmark tests.

Real-world robot forest mapping validates practical effectiveness.

Abstract

Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and observe deterministically. In this paper, we propose DetMCVI, an adaptation of the Monte Carlo Value Iteration (MCVI) algorithm for DetPOMDPs, which builds policies in the form of finite-state controllers (FSCs). DetMCVI solves large problems with a high success rate, outperforming existing baselines for DetPOMDPs. We also verify the performance of the algorithm in a real-world mobile robot forest mapping scenario.