A Simple NLP-based Approach to Support Onboarding and Retention in Open Source Communities

TL;DR

This paper presents a simple NLP-based method to automatically identify issues suitable for newcomers and those likely to be resolved by future active developers in open source projects, enhancing onboarding and retention.

Contribution

It introduces a straightforward NLP approach using supervised classifiers to identify beginner-friendly and future active developer issues from issue descriptions.

Findings

Random Forest achieved 91% precision for identifying beginner issues.

Decision Tree achieved 92% precision for issues resolved by future active developers.

The approach enables automatic issue labeling for onboarding support.

Abstract

Successful open source communities are constantly looking for new members and helping them become active developers. A common approach for developer onboarding in open source projects is to let newcomers focus on relevant yet easy-to-solve issues to familiarize themselves with the code and the community. The goal of this research is twofold. First, we aim at automatically identifying issues that newcomers can resolve by analyzing the history of resolved issues by simply using the title and description of issues. Second, we aim at automatically identifying issues, that can be resolved by newcomers who later become active developers. We mined the issue trackers of three large open source projects and extracted natural language features from the title and description of resolved issues. In a series of experiments, we optimized and compared the accuracy of four supervised classifiers to address our research goals. Random Forest, achieved up to 91% precision (F1-score 72%) towards the first goal while for the second goal, Decision Tree achieved a precision of 92% (F1-score 91%). A qualitative evaluation gave insights on what information in the issue description is helpful for newcomers. Our approach can be used to automatically identify, label, and recommend issues for newcomers in open source software projects based only on the text of the issues.