On the Benefits of Semi-Supervised Test Case Generation for Simulation Models

TL;DR

This paper introduces GenClu, a semi-supervised test case generation system for simulation models that significantly reduces testing time while maintaining or improving test quality compared to fully-supervised methods.

Contribution

The paper presents a novel semi-supervised approach for test case generation in simulation models, demonstrating substantial speedups and comparable or better test effectiveness.

Findings

GenClu is several orders of magnitude faster than previous methods.

Tests generated by GenClu are as effective or better in mutation testing.

Semi-supervised methods outperform fully-supervised approaches in this context.

Abstract

Testing complex simulation models can be expensive and time consuming. Current state-of-the-art methods that explore this problem are fully-supervised; i.e. they require that all examples are labeled. On the other hand, the GenClu system (introduced in this paper) takes a semi-supervised approach; i.e. (a) only a small subset of information is actually labeled (via simulation) and (b) those labels are then spread across the rest of the data. When applied to five open-source simulation models of cyber-physical systems, GenClu's test generation can be multiple orders of magnitude faster than the prior state of the art. Further, when assessed via mutation testing, tests generated by GenClu were as good or better than anything else tested here. Hence, we recommend semi-supervised methods over prior methods (evolutionary search and fully-supervised learning).