HARPS: An Online POMDP Framework for Human-Assisted Robotic Planning and Sensing

TL;DR

HARPS is an innovative framework that combines online POMDP planning, semantic human-robot interaction, and Bayesian data fusion to enhance autonomous robot decision-making in uncertain environments.

Contribution

It introduces a novel integrated approach for active semantic sensing and planning that leverages human input and Bayesian updates within an online POMDP framework.

Findings

Significant reduction in search time and improved belief accuracy in simulations.

Doubling of target capture rate in human-robot experiments.

Robustness of semantic sensing across diverse user interactions.

Abstract

Autonomous robots can benefit greatly from human-provided semantic characterizations of uncertain task environments and states. However, the development of integrated strategies which let robots model, communicate, and act on such 'soft data' remains challenging. Here, the Human Assisted Robotic Planning and Sensing (HARPS) framework is presented for active semantic sensing and planning in human-robot teams to address these gaps by formally combining the benefits of online sampling-based POMDP policies, multimodal semantic interaction, and Bayesian data fusion. This approach lets humans opportunistically impose model structure and extend the range of semantic soft data in uncertain environments by sketching and labeling arbitrary landmarks across the environment. Dynamic updating of the environment model while during search allows robotic agents to actively query humans for novel and relevant semantic data, thereby improving beliefs of unknown environments and states for improved online planning. Simulations of a UAV-enabled target search application in a large-scale partially structured environment show significant improvements in time and belief state estimates required for interception versus conventional planning based solely on robotic sensing. Human subject studies in the same environment (n = 36) demonstrate an average doubling in dynamic target capture rate compared to the lone robot case, and highlight the robustness of active probabilistic reasoning and semantic sensing over a range of user characteristics and interaction modalities.