FipTR: A Simple yet Effective Transformer Framework for Future Instance Prediction in Autonomous Driving

TL;DR

FipTR introduces a streamlined end-to-end transformer framework for future instance prediction in autonomous driving, eliminating complex pipelines and improving accuracy in BEV-based traffic participant forecasting.

Contribution

The paper presents FipTR, a novel transformer-based framework that simplifies future instance prediction by directly estimating masks and incorporating flow-aware attention, reducing reliance on auxiliary outputs.

Findings

FipTR outperforms existing methods in accuracy and efficiency.

The flow-aware BEV predictor enhances temporal coherence.

The approach simplifies the prediction pipeline and improves robustness.

Abstract

The future instance prediction from a Bird's Eye View(BEV) perspective is a vital component in autonomous driving, which involves future instance segmentation and instance motion prediction. Existing methods usually rely on a redundant and complex pipeline which requires multiple auxiliary outputs and post-processing procedures. Moreover, estimated errors on each of the auxiliary predictions will lead to degradation of the prediction performance. In this paper, we propose a simple yet effective fully end-to-end framework named Future Instance Prediction Transformer(FipTR), which views the task as BEV instance segmentation and prediction for future frames. We propose to adopt instance queries representing specific traffic participants to directly estimate the corresponding future occupied masks, and thus get rid of complex post-processing procedures. Besides, we devise a flow-aware BEV predictor for future BEV feature prediction composed of a flow-aware deformable attention that takes backward flow guiding the offset sampling. A novel future instance matching strategy is also proposed to further improve the temporal coherence. Extensive experiments demonstrate the superiority of FipTR and its effectiveness under different temporal BEV encoders. The code is available at https://github.com/TabGuigui/FipTR .