From First Patch to Long-Term Contributor: Evaluating Onboarding Recommendations for OSS Newcomers

TL;DR

This study identifies and evaluates onboarding recommendations for OSS newcomers, revealing which are effective, context-dependent, or detrimental, and providing guidance for fostering long-term OSS contributions.

Contribution

It offers a comprehensive empirical evaluation of 15 onboarding recommendations across diverse OSS projects, highlighting which are beneficial, harmful, or specific to newcomers.

Findings

Four recommendations positively correlate with first patch acceptance.

Four recommendations have context-dependent effects.

Four recommendations show significant negative associations.

Abstract

Attracting and retaining a steady stream of new contributors is crucial to ensuring the long-term survival of open-source software (OSS) projects. However, there are two key research gaps regarding recommendations for onboarding new contributors to OSS projects. First, most of the existing recommendations are based on a limited number of projects, which raises concerns about their generalizability. If a recommendation yields conflicting results in a different context, it could hinder a newcomer's onboarding process rather than help them. Second, it's unclear whether these recommendations also apply to experienced contributors. If certain recommendations are specific to newcomers, continuing to follow them after their initial contributions are accepted could hinder their chances of becoming long-term contributors. To address these gaps, we conducted a two-stage mixed-method study. In the first stage, we conducted a Systematic Literature Review (SLR) and identified 15 task-related actionable recommendations that newcomers to OSS projects can follow to improve their odds of successful onboarding. In the second stage, we conduct a large-scale empirical study of five Gerrit-based projects and 1,155 OSS projects from GitHub to assess whether those recommendations assist newcomers' successful onboarding. Our results suggest that four recommendations positively correlate with newcomers' first patch acceptance in most contexts. Four recommendations are context-dependent, and four indicate significant negative associations for most projects. Our results also found three newcomer-specific recommendations, which OSS joiners should abandon at non-newcomer status to increase their odds of becoming long-term contributors.