Using Source Code Metrics for Predicting Metamorphic Relations at Method Level

TL;DR

This paper explores using source code metrics, beyond control flow graph features, to predict metamorphic relations at the method level, aiming for a less costly yet effective approach.

Contribution

It extends the original PMR approach by incorporating directly extractable source code features and evaluates their effectiveness in predicting metamorphic relations.

Findings

CFG-based features with SVM-RWK perform best overall.

Source code features can outperform CFG features for specific MRs.

The approach achieves high AUC-ROC scores, over 0.8 for some MRs.

Abstract

Metamorphic testing (TM) examines the relations between inputs and outputs of test runs. These relations are known as metamorphic relations (MR). Currently, MRs are handpicked and require in-depth knowledge of the System Under Test (SUT), as well as its problem domain. As a result, the identification and selection of high-quality MRs is a challenge. \citeauthor{PMR1} suggested the Predicting Metamorphic Relations (PMR) approach for automatic prediction of applicable MRs picked from a predefined list. PMR is based on a Support Vector Machine (SVM) model using features derived from the Control Flow Graphs (CFGs) of 100 Java methods. The original study of \citeauthor{PMR1} showed encouraging results, but developing classification models from CFG-related features is costly. In this paper, we aim at developing a PMR approach that is less costly without losing performance. We complement the original PMR approach by considering other than CFG-related features. We define 21 features that can be directly extracted from source code and build several classifiers, including SVM models. Our results indicate that using the original CFG-based method-level features, in particular for a SVM with random walk kernel (RWK), achieve better predictions in terms of AUC-ROC for most of the candidate MRs than our models. However, for one of the candidate MRs, using source code features achieved the best AUC-ROC result (greater than 0.8).