The Necessity of a Holistic Safety Evaluation Framework for AI-Based Automation Features

TL;DR

This paper advocates for a comprehensive safety evaluation framework for AI components in autonomous driving, emphasizing the need to include traditionally non-safety-critical QM components due to AI-related hazards.

Contribution

It highlights the importance of holistic safety analysis for AI components, integrating SOTIF, FuSa, and AI standards to address emerging risks in perception systems.

Findings

AI-driven perception deficiencies can cause safety hazards even in QM components.

Current safety standards may overlook risks from AI components classified as non-safety-critical.

A comprehensive safety approach can better mitigate AI-related hazards in autonomous systems.

Abstract

The intersection of Safety of Intended Functionality (SOTIF) and Functional Safety (FuSa) analysis of driving automation features has traditionally excluded Quality Management (QM) components (components that has no ASIL requirements allocated from vehicle-level HARA) from rigorous safety impact evaluations. While QM components are not typically classified as safety-relevant, recent developments in artificial intelligence (AI) integration reveal that such components can contribute to SOTIF-related hazardous risks. Compliance with emerging AI safety standards, such as ISO/PAS 8800, necessitates re-evaluating safety considerations for these components. This paper examines the necessity of conducting holistic safety analysis and risk assessment on AI components, emphasizing their potential to introduce hazards with the capacity to violate risk acceptance criteria when deployed in safety-critical driving systems, particularly in perception algorithms. Using case studies, we demonstrate how deficiencies in AI-driven perception systems can emerge even in QM-classified components, leading to unintended functional behaviors with critical safety implications. By bridging theoretical analysis with practical examples, this paper argues for the adoption of comprehensive FuSa, SOTIF, and AI standards-driven methodologies to identify and mitigate risks in AI components. The findings demonstrate the importance of revising existing safety frameworks to address the evolving challenges posed by AI, ensuring comprehensive safety assurance across all component classifications spanning multiple safety standards.