Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data

TL;DR

This paper introduces SpAGNN, a spatially-aware graph neural network that models agent interactions for relational behavior forecasting from sensor data, incorporating probabilistic predictions and end-to-end training.

Contribution

The paper presents a novel SpAGNN model that integrates spatially-transformed message passing inspired by Gaussian belief propagation for improved behavior forecasting.

Findings

Achieves significant improvements over state-of-the-art on ATG4D and nuScenes datasets.

Models uncertainty at the trajectory level.

Fully differentiable end-to-end training.

Abstract

In this paper, we tackle the problem of relational behavior forecasting from sensor data. Towards this goal, we propose a novel spatially-aware graph neural network (SpAGNN) that models the interactions between agents in the scene. Specifically, we exploit a convolutional neural network to detect the actors and compute their initial states. A graph neural network then iteratively updates the actor states via a message passing process. Inspired by Gaussian belief propagation, we design the messages to be spatially-transformed parameters of the output distributions from neighboring agents. Our model is fully differentiable, thus enabling end-to-end training. Importantly, our probabilistic predictions can model uncertainty at the trajectory level. We demonstrate the effectiveness of our approach by achieving significant improvements over the state-of-the-art on two real-world self-driving datasets: ATG4D and nuScenes.