Detection of Dataset Shifts in Learning-Enabled Cyber-Physical Systems using Variational Autoencoder for Regression

TL;DR

This paper introduces a novel method using variational autoencoders and conformal anomaly detection to identify dataset shifts in learning-enabled cyber-physical systems, enhancing safety and reliability.

Contribution

It presents a formal framework for dataset shifts in CPS and proposes a new detection approach that considers both input and output data, improving robustness with layer-wise relevance propagation.

Findings

Effective detection of dataset shifts with few false alarms

Detection approach operates faster than system sampling period

Validated on an emergency braking system in a self-driving car simulator

Abstract

Cyber-physical systems (CPSs) use learning-enabled components (LECs) extensively to cope with various complex tasks under high-uncertainty environments. However, the dataset shifts between the training and testing phase may lead the LECs to become ineffective to make large-error predictions, and further, compromise the safety of the overall system. In our paper, we first provide the formal definitions for different types of dataset shifts in learning-enabled CPS. Then, we propose an approach to detect the dataset shifts effectively for regression problems. Our approach is based on the inductive conformal anomaly detection and utilizes a variational autoencoder for regression model which enables the approach to take into consideration both LEC input and output for detecting dataset shifts. Additionally, in order to improve the robustness of detection, layer-wise relevance propagation (LRP) is incorporated into our approach. We demonstrate our approach by using an advanced emergency braking system implemented in an open-source simulator for self-driving cars. The evaluation results show that our approach can detect different types of dataset shifts with a small number of false alarms while the execution time is smaller than the sampling period of the system.