New developer metrics: Are comments as crucial as code contributions?

TL;DR

This paper introduces new developer metrics based on commenting and issue activity, demonstrating their importance and effectiveness in assessing contributions in open-source projects, even with limited data.

Contribution

The study proposes novel, less data-dependent developer metrics emphasizing comments and issues, highlighting their value alongside traditional code contributions.

Findings

Commenting activity is as valuable as code contributions.

Issue-related activity effectively identifies user contributions.

Existence-based metrics provide significant insights despite data sparsity.

Abstract

Open-source code development has become widespread in recent years. As a result, open-source software platforms have also become popular, and millions of developers from diverse locations are able to contribute to the same projects. On these platforms, various knowledge about them is obtained from user activity. This information is used in the form of developer metrics to solve a variety of challenges. In this study, we proposed new developer metrics, including commenting and issue-related activity, that require less information. We concluded that commenting on any feature of a project can be equally as valuable as code contribution. In addition, besides the quantitative ones, metrics based on only the existence of the activity have been shown to offer also considerable results. We saw that issues were crucial in identifying user contributions. Even if a developer makes a contribution to only one issue on a project, the relation between the developer and the project is tight. The hit scores are relatively lower because of the sparsity problem of our dataset; even so, we believe that we have presented improvable and remarkable new developer metrics.