PPTFormer: Pseudo Multi-Perspective Transformer for UAV Segmentation

TL;DR

PPTFormer introduces a pseudo multi-perspective transformer that enhances UAV image segmentation by simulating multiple perspectives without needing multi-view datasets, achieving state-of-the-art results.

Contribution

The paper proposes a novel pseudo multi-perspective learning approach with a transformer architecture, improving UAV segmentation accuracy without multi-view data.

Findings

Achieves state-of-the-art performance on five UAV datasets.

Effectively simulates UAV perspectives through pseudo multi-perspective attention.

Significantly improves segmentation accuracy over existing methods.

Abstract

The ascension of Unmanned Aerial Vehicles (UAVs) in various fields necessitates effective UAV image segmentation, which faces challenges due to the dynamic perspectives of UAV-captured images. Traditional segmentation algorithms falter as they cannot accurately mimic the complexity of UAV perspectives, and the cost of obtaining multi-perspective labeled datasets is prohibitive. To address these issues, we introduce the PPTFormer, a novel \textbf{P}seudo Multi-\textbf{P}erspective \textbf{T}rans\textbf{former} network that revolutionizes UAV image segmentation. Our approach circumvents the need for actual multi-perspective data by creating pseudo perspectives for enhanced multi-perspective learning. The PPTFormer network boasts Perspective Representation, novel Perspective Prototypes, and a specialized encoder and decoder that together achieve superior segmentation results through Pseudo Multi-Perspective Attention (PMP Attention) and fusion. Our experiments demonstrate that PPTFormer achieves state-of-the-art performance across five UAV segmentation datasets, confirming its capability to effectively simulate UAV flight perspectives and significantly advance segmentation precision. This work presents a pioneering leap in UAV scene understanding and sets a new benchmark for future developments in semantic segmentation.