VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction

TL;DR

VectorFlow introduces a novel model that combines image and vector data to improve joint traffic occupancy and flow predictions, achieving top performance in occluded scenarios on the Waymo dataset.

Contribution

The paper presents a new occupancy flow fields predictor that fuses image and vector features using attention modules for enhanced traffic prediction accuracy.

Findings

Achieved 3rd place in Waymo Challenge for occupancy and flow prediction.

Outperformed existing models in occluded occupancy and flow prediction.

Effectively combines rasterized images and continuous trajectories for joint predictions.

Abstract

Predicting future behaviors of road agents is a key task in autonomous driving. While existing models have demonstrated great success in predicting marginal agent future behaviors, it remains a challenge to efficiently predict consistent joint behaviors of multiple agents. Recently, the occupancy flow fields representation was proposed to represent joint future states of road agents through a combination of occupancy grid and flow, which supports efficient and consistent joint predictions. In this work, we propose a novel occupancy flow fields predictor to produce accurate occupancy and flow predictions, by combining the power of an image encoder that learns features from a rasterized traffic image and a vector encoder that captures information of continuous agent trajectories and map states. The two encoded features are fused by multiple attention modules before generating final predictions. Our simple but effective model ranks 3rd place on the Waymo Open Dataset Occupancy and Flow Prediction Challenge, and achieves the best performance in the occluded occupancy and flow prediction task.