Runtime Monitoring DNN-Based Perception
Chih-Hong Cheng, Michael Luttenberger, Rongjie Yan

TL;DR
This paper reviews runtime monitoring techniques for DNN-based perception systems, emphasizing the importance of rigorous monitor design to ensure safety-critical applications operate reliably beyond static verification.
Contribution
It provides an overview of classical and formal methods for runtime verification of DNN perception, highlighting differences in monitor design and the importance of data outside the operational domain.
Findings
Classical and formal methods for runtime monitoring are discussed.
Design of monitors varies between machine learning and formal methods communities.
Data outside the operational domain is crucial for monitor effectiveness.
Abstract
Deep neural networks (DNNs) are instrumental in realizing complex perception systems. As many of these applications are safety-critical by design, engineering rigor is required to ensure that the functional insufficiency of the DNN-based perception is not the source of harm. In addition to conventional static verification and testing techniques employed during the design phase, there is a need for runtime verification techniques that can detect critical events, diagnose issues, and even enforce requirements. This tutorial aims to provide readers with a glimpse of techniques proposed in the literature. We start with classical methods proposed in the machine learning community, then highlight a few techniques proposed by the formal methods community. While we surely can observe similarities in the design of monitors, how the decision boundaries are created vary between the two…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Fault Detection and Control Systems · CCD and CMOS Imaging Sensors
