Identifying Experts in Software Libraries and Frameworks among GitHub Users

TL;DR

This paper evaluates machine learning techniques to identify software developers' expertise in popular JavaScript libraries using GitHub activity data, highlighting challenges and proposing a clustering-based method validated with LinkedIn data.

Contribution

It introduces a clustering-based approach for expert identification and provides a publicly available dataset linking GitHub activity with self-reported expertise.

Findings

Machine learning classifiers face challenges in predicting library expertise.

Clustering combined with LinkedIn data can effectively recommend experts.

A public dataset of 575 developers' expertise is provided.

Abstract

Software development increasingly depends on libraries and frameworks to increase productivity and reduce time-to-market. Despite this fact, we still lack techniques to assess developers expertise in widely popular libraries and frameworks. In this paper, we evaluate the performance of unsupervised (based on clustering) and supervised machine learning classifiers (Random Forest and SVM) to identify experts in three popular JavaScript libraries: facebook/react, mongodb/node-mongodb, and socketio/socket.io. First, we collect 13 features about developers activity on GitHub projects, including commits on source code files that depend on these libraries. We also build a ground truth including the expertise of 575 developers on the studied libraries, as self-reported by them in a survey. Based on our findings, we document the challenges of using machine learning classifiers to predict expertise in software libraries, using features extracted from GitHub. Then, we propose a method to identify library experts based on clustering feature data from GitHub; by triangulating the results of this method with information available on Linkedin profiles, we show that it is able to recommend dozens of GitHub users with evidences of being experts in the studied JavaScript libraries. We also provide a public dataset with the expertise of 575 developers on the studied libraries.