Evaluating Time-Dependent Methods and Seasonal Effects in Code Technical Debt Prediction

TL;DR

This study evaluates the effectiveness of time-dependent and seasonal models in predicting code technical debt, showing that ARIMAX models outperform others and seasonal effects offer modest improvements, with strong industry interest in short- to medium-term forecasts.

Contribution

It introduces the application of time-dependent models, especially ARIMAX, for Code TD prediction and assesses the impact of seasonal effects, filling a gap in standardized prediction approaches.

Findings

ARIMAX outperforms other models in prediction accuracy.

Seasonal effects provide modest improvements in forecasts.

Industry professionals show strong interest in short- to medium-term predictions.

Abstract

Background. Code Technical Debt (Code TD) prediction has gained significant attention in recent software engineering research. However, no standardized approach to Code TD prediction fully captures the factors influencing its evolution. Objective. Our study aims to assess the impact of time-dependent models and seasonal effects on Code TD prediction. It evaluates such models against widely used Machine Learning models, also considering the influence of seasonality on prediction performance. Methods. We trained 11 prediction models with 31 Java open-source projects. To assess their performance, we predicted future observations of the SQALE index. To evaluate the practical usability of our TD forecasting model and its impact on practitioners, we surveyed 23 software engineering professionals. Results. Our study confirms the benefits of time-dependent techniques, with the ARIMAX model outperforming the others. Seasonal effects improved predictive performance, though the impact remained modest. \ReviewerA{ARIMAX/SARIMAX models demonstrated to provide well-balanced long-term forecasts. The survey highlighted strong industry interest in short- to medium-term TD forecasts. Conclusions. Our findings support using techniques that capture time dependence in historical software metric data, particularly for Code TD. Effectively addressing this evidence requires adopting methods that account for temporal patterns.