Testing the Stationarity Assumption in Software Effort Estimation Datasets

TL;DR

This study tests the stationarity assumption in software effort estimation datasets using kernel estimators and finds that non-stationary datasets are better modeled with uniform weights, challenging common assumptions.

Contribution

It introduces a method to test stationarity in effort estimation datasets and compares the effectiveness of uniform versus non-uniform models based on stationarity.

Findings

Non-stationary datasets are better modeled with uniform weights.

Stationarity affects the accuracy of effort estimation models.

Kernel estimator type does not significantly impact model accuracy.

Abstract

Software effort estimation (SEE) models are typically developed based on an underlying assumption that all data points are equally relevant to the prediction of effort for future projects. The dynamic nature of several aspects of the software engineering process could mean that this assumption does not hold in at least some cases. This study employs three kernel estimator functions to test the stationarity assumption in three software engineering datasets that have been used in the construction of software effort estimation models. The kernel estimators are used in the generation of non-uniform weights which are subsequently employed in weighted linear regression modeling. Prediction errors are compared to those obtained from uniform models. Our results indicate that, for datasets that exhibit underlying non-stationary processes, uniform models are more accurate than non-uniform models. In contrast, the accuracy of uniform and non-uniform models for datasets that exhibited stationary processes was essentially equivalent. The results of our study also confirm prior findings that the accuracy of effort estimation models is independent of the type of kernel estimator function used in model development.