A deep learning model for estimating story points

TL;DR

This paper introduces a deep learning model combining LSTM and recurrent highway networks to estimate story points in agile projects, supported by a large dataset and outperforming baseline methods.

Contribution

It presents the first comprehensive dataset for story point estimation and proposes an end-to-end deep learning model that improves accuracy over existing baselines.

Findings

Outperforms baseline effort estimation methods in MAE and accuracy

Uses a novel combination of LSTM and recurrent highway networks

Provides a large dataset for story point estimation in open source projects

Abstract

Although there has been substantial research in software analytics for effort estimation in traditional software projects, little work has been done for estimation in agile projects, especially estimating user stories or issues. Story points are the most common unit of measure used for estimating the effort involved in implementing a user story or resolving an issue. In this paper, we offer for the \emph{first} time a comprehensive dataset for story points-based estimation that contains 23,313 issues from 16 open source projects. We also propose a prediction model for estimating story points based on a novel combination of two powerful deep learning architectures: long short-term memory and recurrent highway network. Our prediction system is \emph{end-to-end} trainable from raw input data to prediction outcomes without any manual feature engineering. An empirical evaluation demonstrates that our approach consistently outperforms three common effort estimation baselines and two alternatives in both Mean Absolute Error and the Standardized Accuracy.