"Estimating software project effort using analogies": Reflections after 28 years

TL;DR

This paper reflects on 28 years of research in software effort estimation using analogy, highlighting progress, enduring issues, and areas for methodological improvement in empirical validation and open science practices.

Contribution

It provides a retrospective analysis of a highly influential paper, emphasizing the evolution and ongoing challenges in software effort prediction research.

Findings

Empirical validation remains crucial in effort estimation.

Open Science principles are underutilized in the field.

Methodological rigor and transparency need enhancement.

Abstract

Background: This invited paper is the result of an invitation to write a retrospective article on a "TSE most influential paper" as part of the journal's 50th anniversary. Objective: To reflect on the progress of software engineering prediction research using the lens of a selected, highly cited research paper and 28 years of hindsight. Methods: The paper examines (i) what was achieved, (ii) what has endured and (iii) what could have been done differently with the benefit of retrospection. Conclusions: While many specifics of software project effort prediction have evolved, key methodological issues remain relevant. The original study emphasised empirical validation with benchmarks, out-of-sample testing and data/tool sharing. Four areas for improvement are identified: (i) stronger commitment to Open Science principles, (ii) focus on effect sizes and confidence intervals, (iii) reporting variability alongside typical results and (iv) more rigorous examination of threats to validity.