Automated Detection of Typed Links in Issue Trackers

TL;DR

This paper investigates the automatic detection of link types in issue trackers using machine learning, demonstrating that a BERT-based model can effectively identify various link types with high accuracy across multiple repositories.

Contribution

The study introduces a BERT-based approach for classifying issue link types, outperforming other models and providing insights into textual features influencing detection accuracy.

Findings

BERT model achieves an average macro F1-score of 0.64 across link types.

High accuracy for Subtask- and Epic- links with F1-scores of 0.89 and 0.97.

Shorter issue texts improve link type prediction accuracy.

Abstract

Stakeholders in software projects use issue trackers like JIRA to capture and manage issues, including requirements and bugs. To ease issue navigation and structure project knowledge, stakeholders manually connect issues via links of certain types that reflect different dependencies, such as Epic-, Block-, Duplicate-, or Relate- links. Based on a large dataset of 15 JIRA repositories, we study how well state-of-the-art machine learning models can automatically detect common link types. We found that a pure BERT model trained on titles and descriptions of linked issues significantly outperforms other optimized deep learning models, achieving an encouraging average macro F1-score of 0.64 for detecting 9 popular link types across all repositories (weighted F1-score of 0.73). For the specific Subtask- and Epic- links, the model achieved top F1-scores of 0.89 and 0.97, respectively. Our model does not simply learn the textual similarity of the issues. In general, shorter issue text seems to improve the prediction accuracy with a strong negative correlation of -0.70. We found that Relate-links often get confused with the other links, which suggests that they are likely used as default links in unclear cases. We also observed significant differences across the repositories, depending on how they are used and by whom.