More Than React: Investigating The Role of EmojiReaction in GitHub Pull Requests

TL;DR

This study investigates how emoji reactions on GitHub pull requests influence review efficiency and developer intentions, revealing that emojis are used for purposes beyond reducing comment noise.

Contribution

It introduces a comprehensive study protocol combining quantitative and qualitative methods to explore emoji reactions' roles and ulterior motives in GitHub reviews.

Findings

Preliminary analysis shows emojis do not always reduce commenting noise.

Developers use emojis with ulterior intentions beyond noise reduction.

The study plans to systematically analyze emoji usage patterns and motivations.

Abstract

Context: Open source software development has become more social and collaborative, especially with the rise of social coding platforms like GitHub. Since 2016, GitHub started to support more informal methods such as emoji reactions, with the goal to reduce commenting noise when reviewing any code changes to a repository. Interestingly, preliminary results indicate that emojis do not always reduce commenting noise (i.e., eight out of 20 emoji reactions), providing evidence that developers use emojis with ulterior intentions. From a reviewing context, the extent to which emoji reactions facilitate for a more efficient review process is unknown. Objective: In this registered report, we introduce the study protocols to investigate ulterior intentions and usages of emoji reactions, apart from reducing commenting noise during the discussions in GitHub pull requests (PRs). As part of the report, we first perform a preliminary analysis to whether emoji reactions can reduce commenting noise in PRs and then introduce the execution plan for the study. Method: We will use a mixed-methods approach in this study, i.e., quantitative and qualitative, with three hypotheses to test.