Breaking the Resource Wall: Geometry-Guided Sequence Modeling for Efficient Semantic Segmentation

TL;DR

This paper introduces DGM-Net, an efficient semantic segmentation architecture that leverages geometric guidance and a novel linear-complexity operator to achieve high accuracy with reduced computational resources.

Contribution

The paper proposes DGM-Net, a resource-efficient architecture utilizing geometric guidance and a new operator, improving segmentation performance without large-scale pretraining or heavy backbones.

Findings

DGM-Net achieves 80.8% mIoU within 28k iterations.

It reaches 82.3% mIoU on Cityscapes test set.

The model maintains stable performance on constrained hardware.

Abstract

High-performance semantic segmentation has achieved significant progress in recent years, often driven by increasingly large backbones and higher computational budgets. While effective, such approaches introduce substantial computational overhead and limit accessibility under constrained hardware settings. In this paper, we propose DGM-Net (Directional Geometric Mamba Network), an efficient architecture that improves modeling capability through structural design rather than increasing model capacity. We introduce Directional Geometric Mamba (G-Mamba), a linear-complexity O(N) operator as an alternative to conventional context modeling modules such as ASPP and PPM. To further enhance structural awareness in state space model (SSM)-based modeling, we design the DGM-Module, which extracts centripetal flow fields and topological skeletons to guide the scanning process and improve boundary preservation. Without relying on large-scale pretraining or heavy backbone scaling, DGM-Net achieves 80.8% mIoU within 28k iterations, 82.3% mIoU on Cityscapes test set, and 45.24% mIoU on ADE20K. In addition, the model maintains stable performance under constrained hardware settings (e.g., batch size of 2 on 8GB VRAM), highlighting its efficiency and practicality. These results demonstrate that incorporating geometric guidance into SSM-based architectures provides an effective and resource-efficient direction for semantic segmentation.