Pilot Performance modeling via observer-based inverse reinforcement learning

TL;DR

This paper introduces an observer-based inverse reinforcement learning approach to model pilot behavior in UAVs, successfully estimating cost functions from observed trajectories with robustness and convergence guarantees.

Contribution

It adapts a novel IRL observer to pilot modeling, enabling convergence to equivalent solutions and demonstrating effectiveness on a quadcopter with a supervisory controller.

Findings

The method converges to one of the IRL solutions.

It accurately learns cost functionals from trajectory data.

The approach is robust in experimental UAV scenarios.

Abstract

The focus of this paper is behavior modeling for pilots of unmanned aerial vehicles. The pilot is assumed to make decisions that optimize an unknown cost functional, which is estimated from observed trajectories using a novel inverse reinforcement learning (IRL) framework. The resulting IRL problem often admits multiple solutions. In this paper, a recently developed novel IRL observer is adapted to the pilot modeling problem. The observer is shown to converge to one of the equivalent solutions of the IRL problem. The developed technique is implemented on a quadcopter where the pilot is a linear quadratic supervisory controller that generates velocity commands for the quadcopter to travel to and hover over a desired location. Experimental results demonstrate the robustness of the method and its ability to learn equivalent cost functionals.