AMMUNet: Multi-Scale Attention Map Merging for Remote Sensing Image Segmentation

TL;DR

AMMUNet introduces a multi-scale attention merging framework with efficient global information capture, significantly improving remote sensing image segmentation accuracy while reducing computational costs.

Contribution

It proposes GMSA and AMMM modules for efficient multi-scale attention merging, enhancing global modeling in remote sensing segmentation.

Findings

Achieved 75.48% mIoU on Vaihingen dataset.

Achieved 77.90% mIoU on Potsdam dataset.

Outperformed existing methods in segmentation accuracy.

Abstract

The advancement of deep learning has driven notable progress in remote sensing semantic segmentation. Attention mechanisms, while enabling global modeling and utilizing contextual information, face challenges of high computational costs and require window-based operations that weaken capturing long-range dependencies, hindering their effectiveness for remote sensing image processing. In this letter, we propose AMMUNet, a UNet-based framework that employs multi-scale attention map merging, comprising two key innovations: the granular multi-head self-attention (GMSA) module and the attention map merging mechanism (AMMM). GMSA efficiently acquires global information while substantially mitigating computational costs in contrast to global multi-head self-attention mechanism. This is accomplished through the strategic utilization of dimension correspondence to align granularity and the reduction of relative position bias parameters, thereby optimizing computational efficiency. The proposed AMMM effectively combines multi-scale attention maps into a unified representation using a fixed mask template, enabling the modeling of global attention mechanism. Experimental evaluations highlight the superior performance of our approach, achieving remarkable mean intersection over union (mIoU) scores of 75.48\% on the challenging Vaihingen dataset and an exceptional 77.90\% on the Potsdam dataset, demonstrating the superiority of our method in precise remote sensing semantic segmentation. Codes are available at https://github.com/interpretty/AMMUNet.