Simulation-based Testing of Simulink Models with Test Sequence and Test Assessment Blocks

TL;DR

This paper introduces HECATE, a novel testing approach for Simulink models that leverages Test Sequence and Test Assessment blocks to guide search-based testing, improving effectiveness and efficiency over existing tools.

Contribution

HECATE uniquely uses information from Simulink's Test Sequence and Test Assessment blocks to enhance search-based testing, outperforming state-of-the-art tools in effectiveness and efficiency.

Findings

HECATE is more effective than S-TALIRO in revealing failures in 94% of benchmark models.

HECATE requires fewer iterations and less computational time in 81% of cases.

Successfully generated a failure-revealing test case for an automotive case study.

Abstract

Simulation-based software testing supports engineers in finding faults in Simulink models. It typically relies on search algorithms that iteratively generate test inputs used to exercise models in simulation to detect design errors. While simulation-based software testing techniques are effective in many practical scenarios, they are typically not fully integrated within the Simulink environment and require additional manual effort. Many techniques require engineers to specify requirements using logical languages that are neither intuitive nor fully supported by Simulink, thereby limiting their adoption in industry. This work presents HECATE, a testing approach for Simulink models using Test Sequence and Test Assessment blocks from Simulink Test. Unlike existing testing techniques, HECATE uses information from Simulink models to guide the search-based exploration. Specifically, HECATE relies on information provided by the Test Sequence and Test Assessment blocks to guide the search procedure. Across a benchmark of 16 Simulink models from different domains and industries, our comparison of HECATE with the state-of-the-art testing tool S-TALIRO indicates that HECATE is both more effective (more failure-revealing test cases) and efficient (less iterations and computational time) than S-TALIRO for ~94% and ~81% of benchmark models respectively. Furthermore, HECATE successfully generated a failure-revealing test case for a representative case study from the automotive domain demonstrating its practical usefulness.