Unifying Runtime Monitoring Approaches for Safety-Critical Machine Learning: Application to Vision-Based Landing

TL;DR

This paper proposes a unified framework for runtime monitoring in safety-critical machine learning, categorizing approaches into three types and demonstrating its application to vision-based landing systems.

Contribution

The paper introduces a comprehensive categorization of runtime monitoring methods, unifying fragmented approaches and enabling systematic evaluation in safety-critical ML applications.

Findings

Framework categorizes monitoring into ODD, OOD, and OMS types.

Demonstrates the framework on an aeronautical landing application.

Facilitates comparison and design of monitoring strategies.

Abstract

Runtime monitoring is essential to ensure the safety of ML applications in safety-critical domains. However, current research is fragmented, with independent methods emerging from different communities. In this paper, we propose a unified framework categorising runtime monitoring approaches into three distinct types: Operational Design Domain (ODD) monitoring, which ensures compliance with expected operating conditions; Out-of-Distribution (OOD) monitoring, which rejects inputs that deviate from the training data; and Out-of-Model-Scope (OMS) monitoring, which detects anomalous model behaviour based its internal states or outputs. We demonstrate the benefits of this categorization with a dedicated experiment on an aeronautical safety-critical application: runway detection during landing. This framework facilitates design of monitoring activities, with complementary categories of monitors, and enables evaluation and comparison of different monitors using common, safety-oriented metrics.