Latent Perspective-Taking via a Schr\"odinger Bridge in Influence-Augmented Local Models

TL;DR

This paper introduces a novel neuro-symbolic framework for robots to understand and predict human mental states in social environments, using influence models and Schr"odinger Bridges for belief transformation, enabling socially-aware decision-making.

Contribution

It proposes a new influence-augmented local model with a Schr"odinger Bridge-based perspective-shift operator for social reasoning in robots, combining structured mental-model learning with belief-space planning.

Findings

Preliminary results in MiniGrid social navigation demonstrate effective social awareness.

The architecture enables synthesis of socially-aware policies via model-based reinforcement learning.

The approach effectively transforms egocentric beliefs into other-centric beliefs for social reasoning.

Abstract

Operating in environments alongside humans requires robots to make decisions under uncertainty. In addition to exogenous dynamics, they must reason over others' hidden mental-models and mental-states. While Interactive POMDPs and Bayesian Theory of Mind formulations are principled, exact nested-belief inference is intractable, and hand-specified models are brittle in open-world settings. We address both by learning structured mental-models and an estimator of others' mental-states. Building on the Influence-Based Abstraction, we instantiate an Influence-Augmented Local Model to decompose socially-aware robot tasks into local dynamics, social influences, and exogenous factors. We propose (a) a neuro-symbolic world model instantiating a factored, discrete Dynamic Bayesian Network, and (b) a perspective-shift operator modeled as an amortized Schr\"odinger Bridge over the learned local dynamics that transports factored egocentric beliefs into other-centric beliefs. We show that this architecture enables agents to synthesize socially-aware policies in model-based reinforcement learning, via decision-time mental-state planning (a Schr\"odinger Bridge in belief space), with preliminary results in a MiniGrid social navigation task.