Identifying the Mood of a Software Development Team by Analyzing Text-Based Communication in Chats with Machine Learning

TL;DR

This paper presents a machine learning approach to analyze text-based communication in software development teams, aiming to assess team mood and communication quality from chat messages.

Contribution

It introduces a novel method for classifying the emotional tone of chat messages in software teams, filling a research gap in analyzing text-based collaboration channels.

Findings

Achieved an average classification accuracy of 62.97% for sentiment analysis.

Demonstrated the approach's ability to provide an overall view of group mood.

Validated the method in an industrial case study with 1947 messages.

Abstract

Software development encompasses many collaborative tasks in which usually several persons are involved. Close collaboration and the synchronization of different members of the development team require effective communication. One established communication channel are meetings which are, however, often not as effective as expected. Several approaches already focused on the analysis of meetings to determine the reasons for inefficiency and dissatisfying meeting outcomes. In addition to meetings, text-based communication channels such as chats and e-mails are frequently used in development teams. Communication via these channels requires a similar appropriate behavior as in meetings to achieve a satisfying and expedient collaboration. However, these channels have not yet been extensively examined in research. In this paper, we present an approach for analyzing interpersonal behavior in text-based communication concerning the conversational tone, the familiarity of sender and receiver, the sender's emotionality, and the appropriateness of the used language. We evaluate our approach in an industrial case study based on 1947 messages sent in a group chat in Zulip over 5.5 months. Using our approach, it was possible to automatically classify written sentences as positive, neutral, or negative with an average accuracy of 62.97% compared to human ratings. Despite this coarse-grained classification, it is possible to gain an overall picture of the adequacy of the textual communication and tendencies in the group mood.