Defining Operational Conditions for Safety-Critical AI-Based Systems from Data

TL;DR

This paper introduces a data-driven, kernel-based method to define and compare operational conditions for safety-critical AI systems, aiding certification and safety assurance.

Contribution

It presents a novel Safety-by-Design approach to automatically derive and compare ODDs from data using a multi-dimensional kernel representation.

Findings

Validated through Monte Carlo simulations and aviation case study.

The data-driven ODD can replicate the original operational conditions.

Automated, deterministic algorithm for ODD representation and comparison.

Abstract

Artificial Intelligence (AI) has been on the rise in many domains, including numerous safety-critical applications. However, for complex systems in the real world, defining the underlying environmental conditions in which the AI-based system must operate -- the Operational Design Domain (ODD) -- is extremely challenging. This often results in an incomplete description of the ODD, which contrasts with the requirements of many domains for certifying AI-based systems. Traditionally, the ODD is created in the early stages of the development process, drawing on sophisticated expert knowledge and related standards. This paper presents a novel Safety-by-Design method to a posteriori define the ODD from previously collected data using a multi-dimensional kernel-based representation. This approach is validated through both Monte Carlo methods and a real-world aviation use case for a future collision-avoidance system. Moreover, by defining under what conditions two ODDs are similar, the paper shows that the data-driven ODD can produce a dataset similar to the original, hidden ODD. Deriving the novel, Safety-by-Design, deterministic kernel-based affinity representation of ODDs is fully automated via a bounded, order-independent algorithm. Utilizing the proposed ODD representation enables future certification of data-driven, safety-critical AI-based systems.