Simulation and Real-World Evaluation of Attack Detection Schemes

TL;DR

This paper develops methods to design and evaluate Dynamic Watermarking attack detectors in cyber-physical systems, demonstrating their effectiveness in real-world and simulated environments compared to classical detectors.

Contribution

It introduces a novel approach to design Dynamic Watermarking detectors with user-defined false alarm rates and evaluates their performance against classical methods.

Findings

Dynamic Watermarking detectors can be tuned for specific false alarm rates.

They detect attacks that classical detectors often miss.

Performance is comparable to classical detectors in attack capability.

Abstract

A variety of anomaly detection schemes have been proposed to detect malicious attacks to Cyber-Physical Systems. Among these schemes, Dynamic Watermarking methods have been proven highly effective at detecting a wide range of attacks. Unfortunately, in contrast to other anomaly detectors, no method has been presented to design a Dynamic Watermarking detector to achieve a user-specified false alarm rate, or subsequently evaluate the capabilities of an attacker under such a selection. This paper describes methods to measure the capability of an attacker, to numerically approximate this metric, and to design a Dynamic Watermarking detector that can achieve a user-specified rate of false alarms. The performance of the Dynamic Watermarking detector is compared to three classical anomaly detectors in simulation and on a real-world platform. These experiments illustrate that the attack capability under the Dynamic Watermarking detector is comparable to those of classic anomaly detectors. Importantly, these experiments also make clear that the Dynamic Watermarking detector is consistently able to detect attacks that the other class of detectors are unable to identify.