Geo-ConvGRU: Geographically Masked Convolutional Gated Recurrent Unit for Bird-Eye View Segmentation

TL;DR

This paper introduces Geo-ConvGRU, a novel module combining geographical masking with convolutional gated recurrent units, to improve temporal dependency modeling in Bird's-Eye View segmentation, achieving state-of-the-art results.

Contribution

The paper proposes Geo-ConvGRU, replacing 3D CNNs with ConvGRU and adding geographical masking, to enhance temporal modeling in Bird's-Eye View segmentation tasks.

Findings

Achieves state-of-the-art performance on NuScenes dataset.

Effectively models long-range temporal dependencies.

Reduces noise in temporal features.

Abstract

Convolutional Neural Networks (CNNs) have significantly impacted various computer vision tasks, however, they inherently struggle to model long-range dependencies explicitly due to the localized nature of convolution operations. Although Transformers have addressed limitations in long-range dependencies for the spatial dimension, the temporal dimension remains underexplored. In this paper, we first highlight that 3D CNNs exhibit limitations in capturing long-range temporal dependencies. Though Transformers mitigate spatial dimension issues, they result in a considerable increase in parameter and processing speed reduction. To overcome these challenges, we introduce a simple yet effective module, Geographically Masked Convolutional Gated Recurrent Unit (Geo-ConvGRU), tailored for Bird's-Eye View segmentation. Specifically, we substitute the 3D CNN layers with ConvGRU in the temporal module to bolster the capacity of networks for handling temporal dependencies. Additionally, we integrate a geographical mask into the Convolutional Gated Recurrent Unit to suppress noise introduced by the temporal module. Comprehensive experiments conducted on the NuScenes dataset substantiate the merits of the proposed Geo-ConvGRU, revealing that our approach attains state-of-the-art performance in Bird's-Eye View segmentation.