Approach Towards Semi-Automated Certification for Low Criticality ML-Enabled Airborne Applications

TL;DR

This paper introduces a semi-automated certification process tailored for low criticality ML systems in aviation, combining manual and automated validation methods to enhance safety assurance.

Contribution

It presents a novel certification framework with structured classification, assurance profiles, and human oversight integration for ML in low criticality airborne applications.

Findings

Effective validation of ML models in aviation context

Enhanced confidence measures for ML system certification

Successful case study with YOLOv8 object detection system

Abstract

As Machine Learning (ML) makes its way into aviation, ML enabled systems including low criticality systems require a reliable certification process to ensure safety and performance. Traditional standards, like DO 178C, which are used for critical software in aviation, do not fully cover the unique aspects of ML. This paper proposes a semi automated certification approach, specifically for low criticality ML systems, focusing on data and model validation, resilience assessment, and usability assurance while integrating manual and automated processes. Key aspects include structured classification to guide certification rigor on system attributes, an Assurance Profile that consolidates evaluation outcomes into a confidence measure the ML component, and methodologies for integrating human oversight into certification activities. Through a case study with a YOLOv8 based object detection system designed to classify military and civilian vehicles in real time for reconnaissance and surveillance aircraft, we show how this approach supports the certification of ML systems in low criticality airborne applications.