# Evaluating Model Testing and Model Checking for Finding Requirements Violations in Simulink Models

## TL;DR

This paper compares model testing and model checking for detecting requirement violations in Simulink models, highlighting their complementary strengths and proposing combined application guidelines.

## Contribution

It introduces an industrial Simulink benchmark, categorizes model types, and evaluates the effectiveness of testing and checking approaches for requirement violation detection.

## Key findings

- Model checking and testing are complementary techniques.
- Combining both approaches improves violation detection.
- Guidelines for integrated application of testing and checking.

## Abstract

Matlab/Simulink is a development and simulation language that is widely used by the Cyber-Physical System (CPS) industry to model dynamical systems. There are two mainstream approaches to verify CPS Simulink models: model testing that attempts to identify failures in models by executing them for a number of sampled test inputs, and model checking that attempts to exhaustively check the correctness of models against some given formal properties. In this paper, we present an industrial Simulink model benchmark, provide a categorization of different model types in the benchmark, describe the recurring logical patterns in the model requirements, and discuss the results of applying model checking and model testing approaches to identify requirements violations in the benchmarked models. Based on the results, we discuss the strengths and weaknesses of model testing and model checking. Our results further suggest that model checking and model testing are complementary and by combining them, we can significantly enhance the capabilities of each of these approaches individually. We conclude by providing guidelines as to how the two approaches can be best applied together.