Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos

TL;DR

This paper introduces Aerial-PASS, a drone-mounted panoramic annular lens system with a lightweight neural network for real-time, high-accuracy scene segmentation, and provides a new annotated dataset for aerial panoramic scenes.

Contribution

It presents the first drone-perspective panoramic scene segmentation dataset and a neural network model optimized for panoramic aerial scene parsing.

Findings

System achieves high accuracy in aerial panoramic scene segmentation.

Model balances segmentation quality with real-time inference speed.

Validated on both public and proprietary aerial datasets.

Abstract

Aerial pixel-wise scene perception of the surrounding environment is an important task for UAVs (Unmanned Aerial Vehicles). Previous research works mainly adopt conventional pinhole cameras or fisheye cameras as the imaging device. However, these imaging systems cannot achieve large Field of View (FoV), small size, and lightweight at the same time. To this end, we design a UAV system with a Panoramic Annular Lens (PAL), which has the characteristics of small size, low weight, and a 360-degree annular FoV. A lightweight panoramic annular semantic segmentation neural network model is designed to achieve high-accuracy and real-time scene parsing. In addition, we present the first drone-perspective panoramic scene segmentation dataset Aerial-PASS, with annotated labels of track, field, and others. A comprehensive variety of experiments shows that the designed system performs satisfactorily in aerial panoramic scene parsing. In particular, our proposed model strikes an excellent trade-off between segmentation performance and inference speed suitable, validated on both public street-scene and our established aerial-scene datasets.