Autonomous sPOMDP Environment Modeling With Partial Model Exploitation

TL;DR

This paper introduces an enhanced sPOMDP-based environment modeling algorithm that improves exploration efficiency and scalability, enabling autonomous robots to generate environment representations more effectively in complex settings.

Contribution

It extends the original surprise-based sPOMDP method, significantly boosting training speed and robustness in environment exploration tasks.

Findings

Increased training speed by 31-63%

Enhanced robustness in less deterministic environments

Demonstrated effectiveness in various simulated environments

Abstract

A state space representation of an environment is a classic and yet powerful tool used by many autonomous robotic systems for efficient and often optimal solution planning. However, designing these representations with high performance is laborious and costly, necessitating an effective and versatile tool for autonomous generation of state spaces for autonomous robots. We present a novel state space exploration algorithm by extending the original surprise-based partially-observable Markov Decision Processes (sPOMDP), and demonstrate its effective long-term exploration planning performance in various environments. Through extensive simulation experiments, we show the proposed model significantly increases efficiency and scalability of the original sPOMDP learning techniques with a range of 31-63% gain in training speed while improving robustness in environments with less deterministic transitions. Our results pave the way for extending sPOMDP solutions to a broader set of environments.