SegRet: An Efficient Design for Semantic Segmentation with Retentive Network

TL;DR

SegRet introduces a lightweight, efficient semantic segmentation model using Retentive Network architecture and a residual decoder, achieving state-of-the-art results with fewer parameters for autonomous driving applications.

Contribution

The paper presents SegRet, a novel semantic segmentation model combining RetNet backbone with a zero-initialized residual decoder for improved efficiency and performance.

Findings

Achieves state-of-the-art performance on ADE20K, Cityscapes, and COCO-Stuff.

Reduces model parameters significantly compared to existing methods.

Maintains high accuracy with lower computational cost.

Abstract

With the rapid evolution of autonomous driving technology and intelligent transportation systems, semantic segmentation has become increasingly critical. Precise interpretation and analysis of real-world environments are indispensable for these advanced applications. However, traditional semantic segmentation approaches frequently face challenges in balancing model performance with computational efficiency, especially regarding the volume of model parameters. To address these constraints, we propose SegRet, a novel model employing the Retentive Network (RetNet) architecture coupled with a lightweight residual decoder that integrates zero-initialization. SegRet offers three distinctive advantages: (1) Lightweight Residual Decoder: by embedding a zero-initialization layer within the residual network structure, the decoder remains computationally streamlined without sacrificing essential information propagation; (2) Robust Feature Extraction: adopting RetNet as its backbone enables SegRet to effectively capture hierarchical image features, thereby enriching the representation quality of extracted features; (3) Parameter Efficiency: SegRet attains state-of-the-art (SOTA) segmentation performance while markedly decreasing the number of parameters, ensuring high accuracy without imposing additional computational burdens. Comprehensive empirical evaluations on prominent benchmarks, such as ADE20K, Citycapes, and COCO-Stuff, highlight the effectiveness and superiority of our method.