Robust Vision-Based Runway Detection through Conformal Prediction and Conformal mAP

TL;DR

This paper applies conformal prediction to vision-based runway detection, providing statistical uncertainty guarantees and introducing a new metric, C-mAP, to enhance safety and reliability in aerospace landing systems.

Contribution

It introduces conformal prediction for uncertainty quantification in runway detection and proposes C-mAP, a novel metric aligning detection performance with statistical guarantees.

Findings

Conformal prediction improves detection reliability with uncertainty quantification.

C-mAP aligns object detection performance with conformal guarantees.

Enhanced safety and potential for certification in aerospace ML systems.

Abstract

We explore the use of conformal prediction to provide statistical uncertainty guarantees for runway detection in vision-based landing systems (VLS). Using fine-tuned YOLOv5 and YOLOv6 models on aerial imagery, we apply conformal prediction to quantify localization reliability under user-defined risk levels. We also introduce Conformal mean Average Precision (C-mAP), a novel metric aligning object detection performance with conformal guarantees. Our results show that conformal prediction can improve the reliability of runway detection by quantifying uncertainty in a statistically sound way, increasing safety on-board and paving the way for certification of ML system in the aerospace domain.