# One evaluation of model-based testing and its automation

## TL;DR

This study evaluates various model-based testing approaches on an automotive network controller, demonstrating that automated and manual model-based tests detect more errors than traditional methods, with automation achieving comparable effectiveness to manual efforts.

## Contribution

It provides an empirical comparison of automated and manual model-based testing techniques, highlighting the effectiveness of automation in error detection within automotive systems.

## Key findings

- Automated model-based test suites detect as many errors as manual ones.
- Model-based testing increases error detection compared to requirement-based testing.
- Increasing model-based tests sixfold yields an 11% increase in detected errors.

## Abstract

Model-based testing relies on behavior models for the generation of model traces: input and expected output---test cases---for an implementation. We use the case study of an automotive network controller to assess different test suites in terms of error detection, model coverage, and implementation coverage. Some of these suites were generated automatically with and without models, purely at random, and with dedicated functional test selection criteria. Other suites were derived manually, with and without the model at hand. Both automatically and manually derived model-based test suites detected significantly more requirements errors than hand-crafted test suites that were directly derived from the requirements. The number of detected programming errors did not depend on the use of models. Automatically generated model-based test suites detected as many errors as hand-crafted model-based suites with the same number of tests. A sixfold increase in the number of model-based tests led to an 11% increase in detected errors.