A Prediction-as-Perception Framework for 3D Object Detection

TL;DR

This paper introduces a biomimetic Prediction-As-Perception framework for 3D object detection that improves accuracy and speed by integrating prediction and perception modules, inspired by how humans observe moving objects.

Contribution

The paper proposes a novel Prediction-As-Perception framework that enhances 3D object detection by combining prediction and perception modules in a biomimetic architecture.

Findings

Improves target tracking accuracy by 10%.

Increases inference speed by 15%.

Reduces computational resource consumption.

Abstract

Humans combine prediction and perception to observe the world. When faced with rapidly moving birds or insects, we can only perceive them clearly by predicting their next position and focusing our gaze there. Inspired by this, this paper proposes the Prediction-As-Perception (PAP) framework, integrating a prediction-perception architecture into 3D object perception tasks to enhance the model's perceptual accuracy. The PAP framework consists of two main modules: prediction and perception, primarily utilizing continuous frame information as input. Firstly, the prediction module forecasts the potential future positions of ego vehicles and surrounding traffic participants based on the perception results of the current frame. These predicted positions are then passed as queries to the perception module of the subsequent frame. The perceived results are iteratively fed back into the prediction module. We evaluated the PAP structure using the end-to-end model UniAD on the nuScenes dataset. The results demonstrate that the PAP structure improves UniAD's target tracking accuracy by 10% and increases the inference speed by 15%. This indicates that such a biomimetic design significantly enhances the efficiency and accuracy of perception models while reducing computational resource consumption.