Assessing the Effectiveness of Attack Detection at a Hackfest on Industrial Control Systems

TL;DR

This paper evaluates the effectiveness of attack detection methods during the SWaT Security Showdown hackfest, highlighting the performance of Water Defense in real-time cyber attack detection on an operational water treatment plant.

Contribution

It provides an analysis of attack detection performance in a real-world hackfest environment and introduces Water Defense as a detection mechanism tested during S3.

Findings

Water Defense successfully detected real-time cyber attacks during S3.

Hackfest provided valuable insights into attack detection effectiveness.

Participation improved understanding of attack and defense dynamics.

Abstract

A hackfest named SWaT Security Showdown (S3) has been organized consecutively for two years. S3 has enabled researchers and practitioners to assess the effectiveness of methods and products aimed at detecting cyber attacks launched in real-time on an operational water treatment plant, namely, Secure Water Treatment (SWaT). In S3 independent attack teams design and launch attacks on SWaT while defence teams protect the plant passively and raise alarms upon attack detection. Attack teams are scored according to how successful they are in performing attacks based on specific intents while the defense teams are scored based on the effectiveness of their methods to detect the attacks. This paper focuses on the first two instances of S3 and summarizes the benefits of hackfest and the performance of an attack detection mechanism, named Water Defense, that was exposed to attackers during S3.