ECMNet:Lightweight Semantic Segmentation with Efficient CNN-Mamba Network

TL;DR

ECMNet is a lightweight semantic segmentation model that combines CNN and Mamba for improved accuracy and efficiency, utilizing novel attention modules and feature fusion techniques.

Contribution

The paper introduces ECMNet, a novel CNN-Mamba hybrid architecture with specialized attention and fusion modules for enhanced segmentation performance.

Findings

Achieves 70.6% mIoU on Cityscapes

Achieves 73.6% mIoU on CamVid

Uses only 0.87M parameters with high efficiency

Abstract

In the past decade, Convolutional Neural Networks (CNNs) and Transformers have achieved wide applicaiton in semantic segmentation tasks. Although CNNs with Transformer models greatly improve performance, the global context modeling remains inadequate. Recently, Mamba achieved great potential in vision tasks, showing its advantages in modeling long-range dependency. In this paper, we propose a lightweight Efficient CNN-Mamba Network for semantic segmentation, dubbed as ECMNet. ECMNet combines CNN with Mamba skillfully in a capsule-based framework to address their complementary weaknesses. Specifically, We design a Enhanced Dual-Attention Block (EDAB) for lightweight bottleneck. In order to improve the representations ability of feature, We devise a Multi-Scale Attention Unit (MSAU) to integrate multi-scale feature aggregation, spatial aggregation and channel aggregation. Moreover, a Mamba enhanced Feature Fusion Module (FFM) merges diverse level feature, significantly enhancing segmented accuracy. Extensive experiments on two representative datasets demonstrate that the proposed model excels in accuracy and efficiency balance, achieving 70.6% mIoU on Cityscapes and 73.6% mIoU on CamVid test datasets, with 0.87M parameters and 8.27G FLOPs on a single RTX 3090 GPU platform.