Safety Assessment for Autonomous Systems' Perception Capabilities

TL;DR

This paper adapts traditional safety analysis methods to address the unique challenges of perception systems in autonomous vehicles, considering ML limitations and environmental factors to improve safety assurance.

Contribution

It introduces a tailored safety-analysis approach for autonomous systems' perception modules, incorporating ML failure modes and environmental effects.

Findings

Proposes a modified safety analysis framework for perception systems.

Demonstrates the method with a partial analysis of a road vehicle.

Discusses applicability to various sensing modalities.

Abstract

Autonomous Systems (AS) are increasingly proposed, or used, in Safety Critical (SC) applications. Many such systems make use of sophisticated sensor suites and processing to provide scene understanding which informs the AS' decision-making. The sensor processing typically makes use of Machine Learning (ML) and has to work in challenging environments, further the ML-algorithms have known limitations,e.g., the possibility of false-negatives or false-positives in object classification. The well-established safety-analysis methods developed for conventional SC systems are not well-matched to AS, ML, or the sensing systems used by AS. This paper proposes an adaptation of well-established safety-analysis methods to address the specifics of perception-systems for AS, including addressing environmental effects and the potential failure-modes of ML, and provides a rationale for choosing particular sets of guidewords, or prompts, for safety-analysis. It goes on to show how the results of the analysis can be used to inform the design and verification of the AS and illustrates the new method by presenting a partial analysis of a road vehicle. Illustrations in the paper are primarily based on optical sensing, however the paper discusses the applicability of the method to other sensing modalities and its role in a wider safety process addressing the overall capabilities of AS.