Increasing System Test Coverage in Production Automation Systems

TL;DR

This paper presents a novel coverage assessment approach for production automation systems, aiding testing technicians in identifying untested behaviors of control software through guided semi-automatic testing and trace analysis, validated in industrial settings.

Contribution

It introduces the first industrially applied coverage assessment method for production automation systems, integrating trace recording and criticality evaluation of untested behaviors.

Findings

Effective identification of untested system behaviors.

Successful application on real industrial systems.

Positive feedback from industrial experts.

Abstract

An approach is introduced, which supports a testing technician in the identification of possibly untested behavior of control software of fully integrated automated production systems (aPS). Based on an approach for guided semi-automatic system testing, execution traces are recorded during testing, allowing a subsequent coverage assessment. As the behavior of an aPS is highly dependent on the software, omitted system behavior can be identified and assessed for criticality. Through close cooperation with industry, this approach represents the first coverage assessment approach for system testing in production automation to be applied on real industrial objects and evaluated by industrial experts.