Prototype-Based Low Altitude UAV Semantic Segmentation

TL;DR

PBSeg is an efficient prototype-based segmentation framework for low-altitude UAV imagery that balances high accuracy with computational efficiency, addressing scale variation and detail capture challenges.

Contribution

The paper introduces PBSeg, a novel prototype-based cross-attention method combined with multi-scale feature extraction for UAV semantic segmentation.

Findings

Achieves 71.86% mIoU on UAVid dataset.

Achieves 80.92% mIoU on UDD6 dataset.

Maintains computational efficiency while capturing fine details.

Abstract

Semantic segmentation of low-altitude UAV imagery presents unique challenges due to extreme scale variations, complex object boundaries, and limited computational resources on edge devices. Existing transformer-based segmentation methods achieve remarkable performance but incur high computational overhead, while lightweight approaches struggle to capture fine-grained details in high-resolution aerial scenes. To address these limitations, we propose PBSeg, an efficient prototype-based segmentation framework tailored for UAV applications. PBSeg introduces a novel prototype-based cross-attention (PBCA) that exploits feature redundancy to reduce computational complexity while maintaining segmentation quality. The framework incorporates an efficient multi-scale feature extraction module that combines deformable convolutions (DConv) with context-aware modulation (CAM) to capture both local details and global semantics. Experiments on two challenging UAV datasets demonstrate the effectiveness of the proposed approach. PBSeg achieves 71.86\% mIoU on UAVid and 80.92\% mIoU on UDD6, establishing competitive performance while maintaining computational efficiency. Code is available at https://github.com/zhangda1018/PBSeg.