Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments

TL;DR

Scene Informer is a unified transformer-based framework that predicts trajectories of observed agents and infers occlusions in partially observable environments, improving autonomous vehicle navigation.

Contribution

It introduces a novel unified approach combining occlusion inference and trajectory prediction using a transformer architecture for partially observable scenes.

Findings

Outperforms existing methods in occupancy prediction.

Achieves better trajectory prediction accuracy.

Effective in partially observable environments.

Abstract

Navigating complex and dynamic environments requires autonomous vehicles (AVs) to reason about both visible and occluded regions. This involves predicting the future motion of observed agents, inferring occluded ones, and modeling their interactions based on vectorized scene representations of the partially observable environment. However, prior work on occlusion inference and trajectory prediction have developed in isolation, with the former based on simplified rasterized methods and the latter assuming full environment observability. We introduce the Scene Informer, a unified approach for predicting both observed agent trajectories and inferring occlusions in a partially observable setting. It uses a transformer to aggregate various input modalities and facilitate selective queries on occlusions that might intersect with the AV's planned path. The framework estimates occupancy probabilities and likely trajectories for occlusions, as well as forecast motion for observed agents. We explore common observability assumptions in both domains and their performance impact. Our approach outperforms existing methods in both occupancy prediction and trajectory prediction in partially observable setting on the Waymo Open Motion Dataset.