ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentation

TL;DR

This paper introduces ViT-BEVSeg, a hierarchical transformer-based network for generating detailed Bird Eye View segmentation maps, showing improved performance over CNN-based methods in autonomous vehicle perception tasks.

Contribution

The paper proposes using vision transformers as a backbone for BEV map generation, replacing traditional CNNs, and demonstrates its effectiveness on the nuScenes dataset.

Findings

Significant performance improvement over CNN-based methods

Effective multi-scale feature representation with vision transformers

Enhanced accuracy in BEV segmentation for autonomous driving

Abstract

Generating a detailed near-field perceptual model of the environment is an important and challenging problem in both self-driving vehicles and autonomous mobile robotics. A Bird Eye View (BEV) map, providing a panoptic representation, is a commonly used approach that provides a simplified 2D representation of the vehicle surroundings with accurate semantic level segmentation for many downstream tasks. Current state-of-the art approaches to generate BEV-maps employ a Convolutional Neural Network (CNN) backbone to create feature-maps which are passed through a spatial transformer to project the derived features onto the BEV coordinate frame. In this paper, we evaluate the use of vision transformers (ViT) as a backbone architecture to generate BEV maps. Our network architecture, ViT-BEVSeg, employs standard vision transformers to generate a multi-scale representation of the input image. The resulting representation is then provided as an input to a spatial transformer decoder module which outputs segmentation maps in the BEV grid. We evaluate our approach on the nuScenes dataset demonstrating a considerable improvement in the performance relative to state-of-the-art approaches.