Inferring World Belief States in Dynamic Real-World Environments

TL;DR

This paper presents methods for robots to infer human belief states in dynamic environments, enabling better teamwork and assistance by modeling internal mental states from observations.

Contribution

It introduces a novel approach to estimate human belief states and team mental models in real-world settings, grounded in mental model theory.

Findings

Successfully inferred human belief states in simulation and real-world robot platforms.

Demonstrated improved active assistance through belief state reasoning.

Extended methods to model team members' beliefs and capabilities.

Abstract

We investigate estimating a human's world belief state using a robot's observations in a dynamic, 3D, and partially observable environment. The methods are grounded in mental model theory, which posits that human decision making, contextual reasoning, situation awareness, and behavior planning draw from an internal simulation or world belief state. When in teams, the mental model also includes a team model of each teammate's beliefs and capabilities, enabling fluent teamwork without the need for constant and explicit communication. In this work we replicate a core component of the team model by inferring a teammate's belief state, or level one situation awareness, as a human-robot team navigates a household environment. We evaluate our methods in a realistic simulation, extend to a real-world robot platform, and demonstrate a downstream application of the belief state through an active assistance semantic reasoning task.