A Trustability Metric for Code Search based on Developer Karma

TL;DR

This paper introduces a trustability metric for code search results that uses developer karma, combining user votes and developer activity to help users identify trustworthy code snippets.

Contribution

It proposes a novel trustability metric based on developer karma and implements it in a code search engine prototype, JBender.

Findings

Preliminary evaluation shows promising trustability ranking.

Developer karma effectively reflects code trustworthiness.

The metric enhances decision-making in code reuse.

Abstract

The promise of search-driven development is that developers will save time and resources by reusing external code in their local projects. To efficiently integrate this code, users must be able to trust it, thus trustability of code search results is just as important as their relevance. In this paper, we introduce a trustability metric to help users assess the quality of code search results and therefore ease the cost-benefit analysis they undertake trying to find suitable integration candidates. The proposed trustability metric incorporates both user votes and cross-project activity of developers to calculate a "karma" value for each developer. Through the karma value of all its developers a project is ranked on a trustability scale. We present JBender, a proof-of-concept code search engine which implements our trustability metric and we discuss preliminary results from an evaluation of the prototype.