What's Really Different with AI? -- A Behavior-based Perspective on System Safety for Automated Driving Systems

TL;DR

This paper discusses the unique safety challenges of AI-based automated driving systems, emphasizing the importance of differentiating AI-specific risks from general uncertainties to improve safety assurance and standardization efforts.

Contribution

It provides guidance on applying relevant standards to AI-driven systems and advocates for separating AI-specific risks from broader uncertainties for better safety analysis.

Findings

Differentiating AI-specific risks enhances safety analysis.

A clear conceptual separation aids in standard application.

Bridging system-level and AI-specific safety analyses improves assurance.

Abstract

Assuring safety for ``AI-based'' systems is one of the current challenges in safety engineering. For automated driving systems, in particular, further assurance challenges result from the open context that the systems need to operate in after deployment. The current standardization and regulation landscape for ``AI-based'' systems is becoming ever more complex, as standards and regulations are being released at high frequencies. This position paper seeks to provide guidance for making qualified arguments which standards should meaningfully be applied to (``AI-based'') automated driving systems. Furthermore, we argue for clearly differentiating sources of risk between AI-specific and general uncertainties related to the open context. In our view, a clear conceptual separation can help to exploit commonalities that can close the gap between system-level and AI-specific safety analyses, while ensuring the required rigor for engineering safe ``AI-based'' systems.