Automated Issue Assignment: Results and Insights from an Industrial Case

TL;DR

This paper reports on the deployment and practical impact of an automated issue assignment system at Softtech/IsBank, highlighting insights from real-world use over several years and emphasizing process adaptations for success.

Contribution

It presents a novel, real-world deployment of automated issue assignment in an industrial setting, with insights on process design and stakeholder management.

Findings

Automated system has been operational since Jan 12, 2018.

Deployment influenced process changes and stakeholder acceptance.

Additional functionalities like online accuracy detection are essential.

Abstract

Softtech, being a subsidiary of the largest private bank in Turkey, called IsBank, receives an average of 350 issue reports from the field every day. Manually assigning the reported issues to the software development teams is costly and cumbersome. We automate the issue assignments using data mining approaches and share our experience gained by deploying the resulting system at Softtech/IsBank. Automated issue assignment has been studied in the literature. However, most of these works report the results obtained on open source projects and the remaining few, although they use commercial, closed source projects, carry out the assignments in a retrospective manner. We, on the other hand, deploy the proposed approach, which has been making all the assignments since Jan 12, 2018. This presents us with an unprecedented opportunity to observe the practical effects of automated issue assignment in the field and to carry out user studies, which have not been done before in this context. We observe that it is not just about deploying a system for automated issue assignment, but also about designing/changing the assignment process around the system; the accuracy of the assignments does not have to be higher than that of manual assignments in order for the system to be useful; deploying such a system requires the development of additional functionalities, such as detecting deteriorations in assignment accuracies in an online manner and creating human-readable explanations for the assignments; stakeholders do not necessarily resist change; and gradual transition can help stakeholders build confidence.