AASeg: Attention Aware Network for Real Time Semantic Segmentation

TL;DR

AASeg is a real-time semantic segmentation network that combines lightweight spatial and channel attention modules with multi-scale context aggregation to improve accuracy without sacrificing speed.

Contribution

It introduces a novel attention-aware architecture with efficient modules for enhanced feature discrimination and contextual understanding in real-time segmentation.

Findings

Outperforms prior real-time segmentation methods on Cityscapes, ADE20K, and CamVid.

Achieves high accuracy with low computational overhead.

Demonstrates a favorable accuracy-speed trade-off.

Abstract

Semantic segmentation is a fundamental task in computer vision that involves dense pixel-wise classification for scene understanding. Despite significant progress, achieving high accuracy while maintaining real-time performance remains a challenging trade-off, particularly for deployment in resource-constrained or latency-sensitive applications. In this paper, we propose AASeg, a novel Attention-Aware Network for real-time semantic segmentation. AASeg effectively captures both spatial and channel-wise dependencies through lightweight Spatial Attention (SA) and Channel Attention (CA) modules, enabling enhanced feature discrimination without incurring significant computational overhead. To enrich contextual representation, we introduce a Multi-Scale Context (MSC) module that aggregates dense local features across multiple receptive fields. The outputs from attention and context modules are adaptively fused to produce high-resolution segmentation maps. Extensive experiments on Cityscapes, ADE20K, and CamVid demonstrate that AASeg achieves a compelling trade-off between accuracy and efficiency, outperforming prior real-time methods.