Behavior-Inspired Neural Networks for Relational Inference

TL;DR

This paper introduces a behavior-inspired neural network model that learns to infer and predict agent relationships and dynamics by mapping observations to latent categories and integrating them into a nonlinear opinion dynamics framework.

Contribution

It proposes a novel abstraction layer for relational inference that captures intermingled categories and enables trajectory prediction and behavior control.

Findings

Effective in learning interpretable relationship categories

Accurate long-horizon trajectory prediction demonstrated

Model naturally identifies mutually exclusive categories

Abstract

From pedestrians to Kuramoto oscillators, interactions between agents govern how dynamical systems evolve in space and time. Discovering how these agents relate to each other has the potential to improve our understanding of the often complex dynamics that underlie these systems. Recent works learn to categorize relationships between agents based on observations of their physical behavior. These approaches model relationship categories as outcomes of a categorical distribution which is limiting and contrary to real-world systems, where relationship categories often intermingle and interact. In this work, we introduce a level of abstraction between the observable behavior of agents and the latent categories that determine their behavior. To do this, we learn a mapping from agent observations to agent preferences for a set of latent categories. The learned preferences and inter-agent proximity are integrated in a nonlinear opinion dynamics model, which allows us to naturally identify mutually exclusive categories, predict an agent's evolution in time, and control an agent's behavior. Through extensive experiments, we demonstrate the utility of our model for learning interpretable categories, and the efficacy of our model for long-horizon trajectory prediction.