Automatically Generating Test Cases for Safety-Critical Software via Symbolic Execution

TL;DR

This paper demonstrates that symbolic execution can effectively generate high-quality test cases for safety-critical software developed in Scade, addressing challenges posed by language constraints and supporting industrial safety standards.

Contribution

It introduces a novel test generator tailored for Scade, showcasing its practical application and effectiveness in a real-world railway safety-critical system.

Findings

Symbolic execution successfully generated test cases for Scade programs.

The approach met safety-critical testing standards in a railway system case study.

Empirical evidence supports symbolic execution as a viable testing method for safety-critical software.

Abstract

Automated test generation based on symbolic execution can be beneficial for systematically testing safety-critical software, to facilitate test engineers to pursue the strict testing requirements mandated by the certification standards, while controlling at the same time the costs of the testing process. At the same time, the development of safety-critical software is often constrained with programming languages or coding conventions that ban linguistic features which are believed to downgrade the safety of the programs, e.g., they do not allow dynamic memory allocation and variable-length arrays, limit the way in which loops are used, forbid recursion, and bound the complexity of control conditions. As a matter of facts, these linguistic features are also the main efficiency-blockers for the test generation approaches based on symbolic execution at the state of the art. This paper contributes new evidence of the effectiveness of generating test cases with symbolic execution for a significant class of industrial safety critical-systems. We specifically focus on Scade, a largely adopted model-based development language for safety-critical embedded software, and we report on a case study in which we exploited symbolic execution to automatically generate test cases for a set of safety-critical programs developed in Scade. To this end, we introduce a novel test generator that we developed in a recent industrial project on testing safety-critical railway software written in Scade, and we report on our experience of using this test generator for testing a set of Scade programs that belong to the development of an on-board signaling unit for high-speed rail. The results provide empirically evidence that symbolic execution is indeed a viable approach for generating high-quality test suites for the safety-critical programs considered in our case study.