TEFormer: Texture-Aware and Edge-Guided Transformer for Semantic Segmentation of Urban Remote Sensing Images

TL;DR

TEFormer is a novel Transformer-based model designed for urban remote sensing image segmentation, utilizing texture and edge information to improve accuracy in complex urban scenes.

Contribution

The paper introduces TEFormer, which integrates a texture-aware module and an edge-guided decoder to enhance semantic segmentation of URSIs, addressing texture and boundary challenges.

Findings

Achieves state-of-the-art mIoU scores on Potsdam and Vaihingen datasets.

Outperforms previous methods by 0.73% and 0.22% in mIoU.

Secures second place on LoveDA dataset with 53.55% mIoU.

Abstract

Accurate semantic segmentation of urban remote sensing images (URSIs) is essential for urban planning and environmental monitoring. However, it remains challenging due to the subtle texture differences and similar spatial structures among geospatial objects, which cause semantic ambiguity and misclassification. Additional complexities arise from irregular object shapes, blurred boundaries, and overlapping spatial distributions of objects, resulting in diverse and intricate edge morphologies. To address these issues, we propose TEFormer, a texture-aware and edge-guided Transformer. Our model features a texture-aware module (TaM) in the encoder to capture fine-grained texture distinctions between visually similar categories, thereby enhancing semantic discrimination. The decoder incorporates an edge-guided tri-branch decoder (Eg3Head) to preserve local edges and details while maintaining multiscale context-awareness. Finally, an edge-guided feature fusion module (EgFFM) effectively integrates contextual, detail, and edge information to achieve refined semantic segmentation. Extensive evaluation demonstrates that TEFormer yields mIoU scores of 88.57% on Potsdam and 81.46% on Vaihingen, exceeding the next best methods by 0.73% and 0.22%. On the LoveDA dataset, it secures the second position with an overall mIoU of 53.55%, trailing the optimal performance by a narrow margin of 0.19%.