A Comparative Study of Programming Languages in Rosetta Code

TL;DR

This study uses Rosetta Code to compare programming languages across various metrics, revealing insights into their conciseness, performance, and reliability based on a large dataset of solutions.

Contribution

It provides a comprehensive, data-driven comparison of eight major programming languages using a large code repository, addressing common debates with empirical evidence.

Findings

Functional and scripting languages are more concise.

C offers the best raw speed on large inputs.

Compiled strongly-typed languages have fewer runtime failures.

Abstract

Sometimes debates on programming languages are more religious than scientific. Questions about which language is more succinct or efficient, or makes developers more productive are discussed with fervor, and their answers are too often based on anecdotes and unsubstantiated beliefs. In this study, we use the largely untapped research potential of Rosetta Code, a code repository of solutions to common programming tasks in various languages, to draw a fair and well-founded comparison. Rosetta Code offers a large data set for analysis. Our study is based on 7087 solution programs corresponding to 745 tasks in 8 widely used languages representing the major programming paradigms (procedural: C and Go; object-oriented: C# and Java; functional: F# and Haskell; scripting: Python and Ruby). Our statistical analysis reveals, most notably, that: functional and scripting languages are more concise than procedural and object-oriented languages; C is hard to beat when it comes to raw speed on large inputs, but performance differences over inputs of moderate size are less pronounced and allow even interpreted languages to be competitive; compiled strongly-typed languages, where more defects can be caught at compile time, are less prone to runtime failures than interpreted or weakly-typed languages. We discuss implications of these results for developers, language designers, and educators.