Readability and Understandability of Snippets Recommended by General-purpose Web Search Engines: a Comparative Study

TL;DR

This study empirically compares the readability and understandability of code snippets from top search engine results, revealing that quality varies by site and rank, with Google generally providing better snippets.

Contribution

It provides a comprehensive empirical analysis of code snippet quality across search engines, highlighting factors influencing readability and understandability.

Findings

Some sites have significantly better readability scores.

Higher-ranked snippets are not always more understandable.

Google's snippets tend to be more readable than Bing or Yahoo!

Abstract

Developers often search for reusable code snippets on general-purpose web search engines like Google, Yahoo! or Microsoft Bing. But some of these code snippets may have poor quality in terms of readability or understandability. In this paper, we propose an empirical analysis to analyze the readability and understandability score from snippets extracted from the web using three independent variables: ranking, general-purpose web search engine, and recommended site. We collected the top-5 recommended sites and their respective code snippet recommendations using Google, Yahoo!, and Bing for 9,480 queries, and evaluate their readability and understandability scores. We found that some recommended sites have significantly better readability and understandability scores than others. The better-ranked code snippet is not necessarily more readable or understandable than a lower-ranked code snippet for all general-purpose web search engines. Moreover, considering the readability score, Google has better-ranked code snippets compared to Yahoo! or Microsoft Bing