Sentiment Polarity Detection for Software Development

TL;DR

This paper introduces Senti4SD, a sentiment classifier tailored for software developers' communication, trained on Stack Overflow data to improve accuracy over general-purpose tools in technical contexts.

Contribution

The paper presents Senti4SD, a novel sentiment analysis tool specifically designed for software engineering, trained on domain-specific data and features to enhance classification accuracy.

Findings

Senti4SD outperforms off-the-shelf tools in classifying developer communications.

It reduces misclassification of neutral and positive posts as negative.

The authors release the classifier and resources for reproducibility.

Abstract

The role of sentiment analysis is increasingly emerging to study software developers' emotions by mining crowd-generated content within social software engineering tools. However, off-the-shelf sentiment analysis tools have been trained on non-technical domains and general-purpose social media, thus resulting in misclassifications of technical jargon and problem reports. Here, we present Senti4SD, a classifier specifically trained to support sentiment analysis in developers' communication channels. Senti4SD is trained and validated using a gold standard of Stack Overflow questions, answers, and comments manually annotated for sentiment polarity. It exploits a suite of both lexicon- and keyword-based features, as well as semantic features based on word embedding. With respect to a mainstream off-the-shelf tool, which we use as a baseline, Senti4SD reduces the misclassifications of neutral and positive posts as emotionally negative. To encourage replications, we release a lab package including the classifier, the word embedding space, and the gold standard with annotation guidelines.