Enhancing Deep Learning with Scenario-Based Override Rules: a Case Study

TL;DR

This paper explores enhancing deep neural networks with scenario-based override rules to improve safety, transparency, and reliability, demonstrated through two case studies and an extended modeling approach.

Contribution

It introduces a novel method of integrating scenario-based override rules with DNNs, improving safety and transparency in neural network systems.

Findings

Feasibility demonstrated through two case studies

Extended scenario-based modeling for better DNN integration

Enhanced safety and transparency of DNN systems

Abstract

Deep neural networks (DNNs) have become a crucial instrument in the software development toolkit, due to their ability to efficiently solve complex problems. Nevertheless, DNNs are highly opaque, and can behave in an unexpected manner when they encounter unfamiliar input. One promising approach for addressing this challenge is by extending DNN-based systems with hand-crafted override rules, which override the DNN's output when certain conditions are met. Here, we advocate crafting such override rules using the well-studied scenario-based modeling paradigm, which produces rules that are simple, extensible, and powerful enough to ensure the safety of the DNN, while also rendering the system more translucent. We report on two extensive case studies, which demonstrate the feasibility of the approach; and through them, propose an extension to scenario-based modeling, which facilitates its integration with DNN components. We regard this work as a step towards creating safer and more reliable DNN-based systems and models.