FB-BEV: BEV Representation from Forward-Backward View Transformations

TL;DR

FB-BEV introduces a novel view transformation module that combines forward and backward projections to improve BEV perception, achieving state-of-the-art results on nuScenes dataset.

Contribution

The paper proposes a new forward-backward view transformation module that enhances BEV feature quality by leveraging the strengths of both existing paradigms.

Findings

Achieves 62.4% NDS on nuScenes test set.

Outperforms existing BEV perception methods.

Provides a unified framework for view transformation.

Abstract

View Transformation Module (VTM), where transformations happen between multi-view image features and Bird-Eye-View (BEV) representation, is a crucial step in camera-based BEV perception systems. Currently, the two most prominent VTM paradigms are forward projection and backward projection. Forward projection, represented by Lift-Splat-Shoot, leads to sparsely projected BEV features without post-processing. Backward projection, with BEVFormer being an example, tends to generate false-positive BEV features from incorrect projections due to the lack of utilization on depth. To address the above limitations, we propose a novel forward-backward view transformation module. Our approach compensates for the deficiencies in both existing methods, allowing them to enhance each other to obtain higher quality BEV representations mutually. We instantiate the proposed module with FB-BEV, which achieves a new state-of-the-art result of 62.4% NDS on the nuScenes test set. Code and models are available at https://github.com/NVlabs/FB-BEV.