Multi-Agent Variational Occlusion Inference Using People as Sensors

TL;DR

This paper introduces a multimodal occlusion inference method for autonomous vehicles that models driver behaviors as sensors, using a variational autoencoder to handle uncertainty and fuse multiple agent observations for safer navigation.

Contribution

It proposes a novel variational autoencoder-based approach to infer occupancy from driver behaviors, explicitly modeling multimodality and aleatoric uncertainty in multi-agent scenarios.

Findings

Outperforms baseline methods in real-world intersection tests.

Handles multi-agent sensor fusion effectively.

Operates in real-time for autonomous driving applications.

Abstract

Autonomous vehicles must reason about spatial occlusions in urban environments to ensure safety without being overly cautious. Prior work explored occlusion inference from observed social behaviors of road agents, hence treating people as sensors. Inferring occupancy from agent behaviors is an inherently multimodal problem; a driver may behave similarly for different occupancy patterns ahead of them (e.g., a driver may move at constant speed in traffic or on an open road). Past work, however, does not account for this multimodality, thus neglecting to model this source of aleatoric uncertainty in the relationship between driver behaviors and their environment. We propose an occlusion inference method that characterizes observed behaviors of human agents as sensor measurements, and fuses them with those from a standard sensor suite. To capture the aleatoric uncertainty, we train a conditional variational autoencoder with a discrete latent space to learn a multimodal mapping from observed driver trajectories to an occupancy grid representation of the view ahead of the driver. Our method handles multi-agent scenarios, combining measurements from multiple observed drivers using evidential theory to solve the sensor fusion problem. Our approach is validated on a cluttered, real-world intersection, outperforming baselines and demonstrating real-time capable performance. Our code is available at https://github.com/sisl/MultiAgentVariationalOcclusionInference .