Finding Causally Different Tests for an Industrial Control System

TL;DR

This paper introduces a guided fuzzing method to generate causally diverse tests for industrial control systems, enhancing testing coverage by identifying meaningfully different manipulations of sensor and actuator inputs.

Contribution

It presents a formal framework and algorithm for finding causally different tests in ICSs, outperforming existing fuzzers in diversity of generated tests.

Findings

Found 106% more causally different tests than comparable fuzzers.

Formalized sensor/actuator manipulation strategies for ICS testing.

Demonstrated effectiveness on a real-world water treatment system.

Abstract

Industrial control systems (ICSs) are types of cyber-physical systems in which programs, written in languages such as ladder logic or structured text, control industrial processes through sensing and actuating. Given the use of ICSs in critical infrastructure, it is important to test their resilience against manipulations of sensor/actuator inputs. Unfortunately, existing methods fail to test them comprehensively, as they typically focus on finding the simplest-to-craft manipulations for a testing goal, and are also unable to determine when a test is simply a minor permutation of another, i.e. based on the same causal events. In this work, we propose a guided fuzzing approach for finding 'meaningfully different' tests for an ICS via a general formalisation of sensor/actuator-manipulation strategies. Our algorithm identifies the causal events in a test, generalises them to an equivalence class, and then updates the fuzzing strategy so as to find new tests that are causally different from those already identified. An evaluation of our approach on a real-world water treatment system shows that it is able to find 106% more causally different tests than the most comparable fuzzer. While we focus on diversifying the test suite of an ICS, our formalisation may be useful for other fuzzers that intercept communication channels.