Impacts of Bad ESP (Early Size Predictions) on Software Effort Estimation

TL;DR

This study investigates how inaccuracies in early size predictions affect software effort estimation, finding that such errors have a surprisingly minor impact compared to other sources of estimation error.

Contribution

The paper demonstrates through analysis and simulation that errors in early size predictions minimally influence overall effort estimation accuracy.

Findings

ESP errors up to +/- 100% have little effect on effort estimates

Effort estimation errors are dominated by other factors, not ESP inaccuracies

Models are robust against significant early size prediction errors

Abstract

Context: Early size predictions (ESP) can lead to errors in effort predictions for software projects. This problem is particular acute in parametric effort models that give extra weight to size factors (for example, the COCOMO model assumes that effort is exponentially proportional to project size). Objective: To test if effort estimates are crippled by bad ESP. Method: Document inaccuracies in early size estimates. Use those error sizes to determine the implications of those inaccuracies via a Monte Carlo perturbation analysis of effort models and an analysis of the equations used in those effort models. Results: While many projects have errors in ESP of up to +/- 100%, those errors add very little to the overall effort estimate error. Specifically, we find no statistically significant difference in the estimation errors seen after increasing ESP errors from 0 to +/- 100%. An analysis of effort estimation models explains why this is so: the net additional impact of ESP error is relatively small compared to the other sources of error associated with in estimation models. Conclusion: ESP errors effect effort estimates by a relatively minor amount. As soon as a model uses a size estimate and other factors to predict project effort, then ESP errors are not crippling to the process of estimation