LSSInst: Improving Geometric Modeling in LSS-Based BEV Perception with Instance Representation

TL;DR

LSSInst enhances BEV perception in camera-only 3D detection by integrating instance representations with existing LSS frameworks, improving detail retention and overall accuracy in autonomous driving scenarios.

Contribution

The paper introduces a two-stage detector with an instance adaptor that improves geometric detail preservation in LSS-based BEV perception methods.

Findings

Boosts performance on nuScenes benchmark

Enhances geometric detail retention in BEV representations

Outperforms current LSS-based state-of-the-art methods

Abstract

With the attention gained by camera-only 3D object detection in autonomous driving, methods based on Bird-Eye-View (BEV) representation especially derived from the forward view transformation paradigm, i.e., lift-splat-shoot (LSS), have recently seen significant progress. The BEV representation formulated by the frustum based on depth distribution prediction is ideal for learning the road structure and scene layout from multi-view images. However, to retain computational efficiency, the compressed BEV representation such as in resolution and axis is inevitably weak in retaining the individual geometric details, undermining the methodological generality and applicability. With this in mind, to compensate for the missing details and utilize multi-view geometry constraints, we propose LSSInst, a two-stage object detector incorporating BEV and instance representations in tandem. The proposed detector exploits fine-grained pixel-level features that can be flexibly integrated into existing LSS-based BEV networks. Having said that, due to the inherent gap between two representation spaces, we design the instance adaptor for the BEV-to-instance semantic coherence rather than pass the proposal naively. Extensive experiments demonstrated that our proposed framework is of excellent generalization ability and performance, which boosts the performances of modern LSS-based BEV perception methods without bells and whistles and outperforms current LSS-based state-of-the-art works on the large-scale nuScenes benchmark.