Peaking into the Black-box: Prediction Intervals Give Insight into Data-driven Quadrotor Model Reliability

TL;DR

This paper evaluates prediction intervals for data-driven quadrotor models, demonstrating their ability to quantify uncertainty and reliability, especially during extrapolation, using polynomial and neural network models validated on simulations and real flight data.

Contribution

It introduces and validates prediction interval estimation techniques for quadrotor models, highlighting their effectiveness in assessing model reliability under uncertainty.

Findings

ANN prediction intervals widen during extrapolation

Polynomial PIs are less sensitive to extrapolation

PIs provide probabilistic bounds for model outputs

Abstract

Ensuring the reliability and validity of data-driven quadrotor model predictions is essential for their accepted and practical use. This is especially true for grey- and black-box models wherein the mapping of inputs to predictions is not transparent and subsequent reliability notoriously difficult to ascertain. Nonetheless, such techniques are frequently and successfully used to identify quadrotor models. Prediction intervals (PIs) may be employed to provide insight into the consistency and accuracy of model predictions. This paper estimates such PIs for polynomial and Artificial Neural Network (ANN) quadrotor aerodynamic models. Two existing ANN PI estimation techniques - the bootstrap method and the quality driven method - are validated numerically for quadrotor aerodynamic models using an existing high-fidelity quadrotor simulation. Quadrotor aerodynamic models are then identified on real quadrotor flight data to demonstrate their utility and explore their sensitivity to model interpolation and extrapolation. It is found that the ANN-based PIs widen considerably when extrapolating and remain constant, or shrink, when interpolating. While this behaviour also occurs for the polynomial PIs, it is of lower magnitude. The estimated PIs establish probabilistic bounds within which the quadrotor model outputs will likely lie, subject to modelling and measurement uncertainties that are reflected through the PI widths.