Learning a Behavior Model of Hybrid Systems Through Combining Model-Based Testing and Machine Learning (Full Version)

TL;DR

This paper presents a method combining automata learning and model-based testing to automatically generate training data for hybrid system models, significantly improving neural network crash detection accuracy with less data.

Contribution

It introduces a novel approach that integrates automata learning and testing to efficiently generate training data for hybrid system models, enhancing machine learning performance.

Findings

Recurrent neural networks achieved five times lower crash detection error.

F1-score comparable with up to 1000 times fewer training samples.

Method outperforms models trained on randomly generated data.

Abstract

Models play an essential role in the design process of cyber-physical systems. They form the basis for simulation and analysis and help in identifying design problems as early as possible. However, the construction of models that comprise physical and digital behavior is challenging. Therefore, there is considerable interest in learning such hybrid behavior by means of machine learning which requires sufficient and representative training data covering the behavior of the physical system adequately. In this work, we exploit a combination of automata learning and model-based testing to generate sufficient training data fully automatically. Experimental results on a platooning scenario show that recurrent neural networks learned with this data achieved significantly better results compared to models learned from randomly generated data. In particular, the classification error for crash detection is reduced by a factor of five and a similar F1-score is obtained with up to three orders of magnitude fewer training samples.