FlyPose: Towards Robust Human Pose Estimation From Aerial Views

TL;DR

FlyPose is a lightweight, real-time human pose estimation system designed for aerial views, significantly improving detection accuracy and capable of operating onboard UAVs in challenging conditions.

Contribution

The paper introduces FlyPose, a novel lightweight aerial human pose estimation pipeline with multi-dataset training and a new challenging dataset, FlyPose-104.

Findings

6.8 mAP improvement in person detection across multiple datasets

16.3 mAP improvement on UAV-Human pose estimation

Operates with ~20 ms latency on Jetson Orin AGX

Abstract

Unmanned Aerial Vehicles (UAVs) are increasingly deployed in close proximity to humans for applications such as parcel delivery, traffic monitoring, disaster response and infrastructure inspections. Ensuring safe and reliable operation in these human-populated environments demands accurate perception of human poses and actions from an aerial viewpoint. This perspective challenges existing methods with low resolution, steep viewing angles and (self-)occlusion, especially if the application demands realtime feasibile models. We train and deploy FlyPose, a lightweight top-down human pose estimation pipeline for aerial imagery. Through multi-dataset training, we achieve an average improvement of 6.8 mAP in person detection across the test-sets of Manipal-UAV, VisDrone, HIT-UAV as well as our custom dataset. For 2D human pose estimation we report an improvement of 16.3 mAP on the challenging UAV-Human dataset. FlyPose runs with an inference latency of ~20 milliseconds including preprocessing on a Jetson Orin AGX Developer Kit and is deployed onboard a quadrotor UAV during flight experiments. We also publish FlyPose-104, a small but challenging aerial human pose estimation dataset, that includes manual annotations from difficult aerial perspectives: https://github.com/farooqhassaan/FlyPose.