Using High-Level Patterns to Estimate How Humans Predict a Robot will Behave

TL;DR

This paper introduces a novel approach for robots to estimate human predictions of robot behavior by using high-level behavior patterns embedded in a discrete latent space, improving interaction safety.

Contribution

It develops a second-order theory of mind model that captures human high-level behavior predictions from trajectory data, enabling robots to better understand human expectations.

Findings

The model's predictions align with actual human expectations in simulations.

Initial evidence shows the approach works with real user data.

The method improves understanding of human predictions in interactive driving scenarios.

Abstract

Humans interacting with robots often form predictions of what the robot will do next. For instance, based on the recent behavior of an autonomous car, a nearby human driver might predict that the car is going to remain in the same lane. It is important for the robot to understand the human's prediction for safe and seamless interaction: e.g., if the autonomous car knows the human thinks it is not merging -- but the autonomous car actually intends to merge -- then the car can adjust its behavior to prevent an accident. Prior works typically assume that humans make precise predictions of robot behavior. However, recent research on human-human prediction suggests the opposite: humans tend to approximate other agents by predicting their high-level behaviors. We apply this finding to develop a second-order theory of mind approach that enables robots to estimate how humans predict they will behave. To extract these high-level predictions directly from data, we embed the recent human and robot trajectories into a discrete latent space. Each element of this latent space captures a different type of behavior (e.g., merging in front of the human, remaining in the same lane) and decodes into a vector field across the state space that is consistent with the underlying behavior type. We hypothesize that our resulting high-level and course predictions of robot behavior will correspond to actual human predictions. We provide initial evidence in support of this hypothesis through proof-of-concept simulations, testing our method's predictions against those of real users, and experiments on a real-world interactive driving dataset.