Runtime Safety Monitoring of Deep Neural Networks for Perception: A Survey

TL;DR

This survey reviews runtime safety monitoring techniques for deep neural networks in perception systems, focusing on detecting safety issues like OOD inputs and adversarial attacks during inference without altering the models.

Contribution

It categorizes and analyzes existing runtime safety monitoring methods for DNNs, identifying strengths, limitations, and future research directions.

Findings

Monitoring inputs, internal representations, and outputs are key categories.

Current methods effectively detect safety concerns but face limitations in robustness.

Open challenges include handling complex safety scenarios and real-time constraints.

Abstract

Deep neural networks (DNNs) are widely used in perception systems for safety-critical applications, such as autonomous driving and robotics. However, DNNs remain vulnerable to various safety concerns, including generalization errors, out-of-distribution (OOD) inputs, and adversarial attacks, which can lead to hazardous failures. This survey provides a comprehensive overview of runtime safety monitoring approaches, which operate in parallel to DNNs during inference to detect these safety concerns without modifying the DNN itself. We categorize existing methods into three main groups: Monitoring inputs, internal representations, and outputs. We analyze the state-of-the-art for each category, identify strengths and limitations, and map methods to the safety concerns they address. In addition, we highlight open challenges and future research directions.