Comparison of Machine Learning Methods for Assigning Software Issues to Team Members

TL;DR

This paper compares shallow and deep machine learning methods for assigning software issues to roles within teams, introducing new features, datasets, and a focus on role-based classification to improve industrial issue management.

Contribution

It introduces a comprehensive feature set, a novel role-based label scheme, and publicly shares industrial datasets, demonstrating the effectiveness of ensemble shallow classifiers comparable to deep models.

Findings

Ensemble shallow classifiers achieve 0.92 accuracy in issue assignment.

Five industrial datasets with 5324 issues are publicly shared.

Shallow methods perform comparably to deep language models in this task.

Abstract

Software issues contain units of work to fix, improve, or create new threads during the development and facilitate communication among the team members. Assigning an issue to the most relevant team member and determining a category of an issue is a tedious and challenging task. Wrong classifications cause delays and rework in the project and trouble among the team members. This paper proposes a set of carefully curated linguistic features for shallow machine learning methods and compares the performance of shallow and ensemble methods with deep language models. Unlike the state-of-the-art, we assign issues to four roles (designer, developer, tester, and leader) rather than to specific individuals or teams to contribute to the generality of our solution. We also consider the level of experience of the developers to reflect the industrial practices in our solution formulation. We collect and annotate five industrial data sets from one of the top three global television producers to evaluate our proposal and compare it with deep language models. Our data sets contain 5324 issues in total. We show that an ensemble classifier of shallow techniques achieves 0.92 for issue assignment in accuracy which is statistically comparable to the state-of-the-art deep language models. The contributions include the public sharing of five annotated industrial issue data sets, the development of a clear and comprehensive feature set, the introduction of a novel label set, and the validation of the efficacy of an ensemble classifier of shallow machine learning techniques.