Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs

TL;DR

This paper introduces a graphical model framework that unifies object states, robot knowledge, and human beliefs, enabling robots to better understand human false-beliefs and improve multi-view perception accuracy.

Contribution

It proposes a novel joint inference method over parse graphs from multiple robots and views to enhance reasoning about human beliefs and object states.

Findings

Accurately recognizes human false-beliefs in various scenarios.

Achieves improved cross-view accuracy on small object tracking.

Effectively fuses multi-view parse graphs for better inference.

Abstract

Aiming to understand how human (false-)belief--a core socio-cognitive ability--would affect human interactions with robots, this paper proposes to adopt a graphical model to unify the representation of object states, robot knowledge, and human (false-)beliefs. Specifically, a parse graph (pg) is learned from a single-view spatiotemporal parsing by aggregating various object states along the time; such a learned representation is accumulated as the robot's knowledge. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning and inference capability to overcome the errors originated from a single view. In the experiments, through the joint inference over pg-s, the system correctly recognizes human (false-)belief in various settings and achieves better cross-view accuracy on a challenging small object tracking dataset.