BEVANet: Bilateral Efficient Visual Attention Network for Real-Time Semantic Segmentation

TL;DR

BEVANet is a novel real-time semantic segmentation network that combines bilateral visual attention mechanisms, large kernel modules, and boundary-aware fusion to achieve high accuracy and efficiency without pretraining.

Contribution

It introduces the Large Kernel Attention mechanism and a bilateral architecture with adaptive feature fusion, advancing real-time segmentation performance.

Findings

Achieves 79.3% mIoU without pretraining

Runs at 33 FPS in real-time

Outperforms existing methods on Cityscapes dataset

Abstract

Real-time semantic segmentation presents the dual challenge of designing efficient architectures that capture large receptive fields for semantic understanding while also refining detailed contours. Vision transformers model long-range dependencies effectively but incur high computational cost. To address these challenges, we introduce the Large Kernel Attention (LKA) mechanism. Our proposed Bilateral Efficient Visual Attention Network (BEVANet) expands the receptive field to capture contextual information and extracts visual and structural features using Sparse Decomposed Large Separable Kernel Attentions (SDLSKA). The Comprehensive Kernel Selection (CKS) mechanism dynamically adapts the receptive field to further enhance performance. Furthermore, the Deep Large Kernel Pyramid Pooling Module (DLKPPM) enriches contextual features by synergistically combining dilated convolutions and large kernel attention. The bilateral architecture facilitates frequent branch communication, and the Boundary Guided Adaptive Fusion (BGAF) module enhances boundary delineation by integrating spatial and semantic features under boundary guidance. BEVANet achieves real-time segmentation at 33 FPS, yielding 79.3% mIoU without pretraining and 81.0% mIoU on Cityscapes after ImageNet pretraining, demonstrating state-of-the-art performance. The code and model is available at https://github.com/maomao0819/BEVANet.