The eyes and hearts of UAV pilots: observations of physiological responses in real-life scenarios

TL;DR

This paper presents a novel method combining object detection, gaze, and heart-rate data to monitor UAV pilots' physiological responses during real-life outdoor flights, aiding training and performance enhancement.

Contribution

It introduces an integrated system for real-time physiological and behavioral monitoring of UAV pilots in outdoor scenarios, addressing limitations of traditional simulation training.

Findings

Pilots' physiological responses vary across flight challenges.

The system effectively captures gaze and heart-rate variability during flights.

Potential to improve pilot training and safety through physiological feedback.

Abstract

The drone industry is diversifying and the number of pilots increases rapidly. In this context, flight schools need adapted tools to train pilots, most importantly with regard to their own awareness of their physiological and cognitive limits. In civil and military aviation, pilots can train themselves on realistic simulators to tune their reaction and reflexes, but also to gather data on their piloting behavior and physiological states. It helps them to improve their performances. Opposed to cockpit scenarios, drone teleoperation is conducted outdoor in the field, thus with only limited potential from desktop simulation training. This work aims to provide a solution to gather pilots behavior out in the field and help them increase their performance. We combined advance object detection from a frontal camera to gaze and heart-rate variability measurements. We observed pilots and analyze their behavior over three flight challenges. We believe this tool can support pilots both in their training and in their regular flight tasks. A demonstration video is available on https://www.youtube.com/watch?v=eePhjd2qNiI