Classifying Multilingual User Feedback using Traditional Machine Learning and Deep Learning

TL;DR

This paper compares traditional machine learning and deep learning methods for classifying multilingual user feedback into categories like problem reports and inquiries, demonstrating that traditional methods can achieve comparable results with large labeled datasets.

Contribution

It provides a comparative analysis of traditional machine learning and deep learning for multilingual user feedback classification, highlighting the effectiveness of traditional methods.

Findings

Traditional machine learning achieves comparable accuracy to deep learning.

Large labeled datasets enable effective classification with traditional methods.

Multilingual feedback classification is feasible with both approaches.

Abstract

With the rise of social media like Twitter and of software distribution platforms like app stores, users got various ways to express their opinion about software products. Popular software vendors get user feedback thousandfold per day. Research has shown that such feedback contains valuable information for software development teams such as problem reports or feature and support inquires. Since the manual analysis of user feedback is cumbersome and hard to manage many researchers and tool vendors suggested to use automated analyses based on traditional supervised machine learning approaches. In this work, we compare the results of traditional machine learning and deep learning in classifying user feedback in English and Italian into problem reports, inquiries, and irrelevant. Our results show that using traditional machine learning, we can still achieve comparable results to deep learning, although we collected thousands of labels.