Compiler Testing: A Systematic Literature Analysis

TL;DR

This paper systematically analyzes the literature on compiler testing, revealing key contributors, popular testing techniques like random testing, and collaboration patterns among researchers to better understand the field.

Contribution

It introduces a comprehensive literature analysis framework combining bibliometric and association rule analysis for compiler testing research.

Findings

USA leads in influential researchers and institutions

'Random testing' is the most active keyword

Researchers tend to collaborate within small groups

Abstract

Compilers are widely-used infrastructures in accelerating the software development, and expected to be trustworthy. In the literature, various testing technologies have been proposed to guarantee the quality of compilers. However, there remains an obstacle to comprehensively characterize and understand compiler testing. To overcome this obstacle, we propose a literature analysis framework to gain insights into the compiler testing area. First, we perform an extensive search to construct a dataset related to compiler testing papers. Then, we conduct a bibliometric analysis to analyze the productive authors, the influential papers, and the frequently tested compilers based on our dataset. Finally, we utilize association rules and collaboration networks to mine the authorships and the communities of interests among researchers and keywords. Some valuable results are reported. We find that the USA is the leading country that contains the most influential researchers and institutions. The most active keyword is "random testing". We also find that most researchers have broad interests within small-scale collaborators in the compiler testing area.