Search-based Testing of Simulink Models with Requirements Tables

TL;DR

This paper introduces the first black-box search-based testing approach for Simulink models that supports Requirements Tables, enabling more practical and industry-friendly testing without relying on logical requirement expressions.

Contribution

The work presents a novel SBST method for Simulink models that integrates with Requirements Tables, improving adoption and effectiveness in industrial settings.

Findings

Achieved 70% success rate in finding failure-revealing tests for model-RT pairs.

Identified failure cases in an industrial cruise controller model that other tools missed.

Maintained efficiency comparable to existing SBST tools not using Requirements Tables.

Abstract

Search-based software testing (SBST) of Simulink models helps find scenarios that demonstrate that the system can reach a state that violates one of its requirements. However, many SBST techniques for Simulink models rely on requirements being expressed in logical languages, limiting their adoption in industry. To help with the adoption, SBST methods and tools for Simulink models need to be integrated with tools used by engineers to specify requirements. This work presents the first black-box testing approach for Simulink models that supports Requirements Table (RT), a tool from Simulink Requirements Toolbox used by practitioners to express software requirements. We evaluated our solution by considering 60 model-RT combinations each made by a model and an RT. Our SBST framework returned a failure-revealing test case for 70% of the model-RT combinations. Remarkably, it identified a failure-revealing test case for three model-RT combinations for a cruise controller of an industrial simulator that other previously used tools were not able to find. The efficiency of our SBST solution is acceptable for practical applications and comparable with existing SBST tools that are not based on RT.