An Analysis of Research in Software Engineering: Assessment and Trends

TL;DR

This paper introduces an automated assessment method for software engineering research, analyzing publication trends and identifying top scholars, institutions, and research directions from 2008 to 2013.

Contribution

It presents a new evaluation module that considers publication quality, author roles, and rankings, improving the assessment of scholars and institutions in software engineering.

Findings

Research subjects follow power-law distributions, indicating the Matthew Effect.

Top scholar: Mark Harman; top institution: University of California; top country: USA.

Identifies research trend changes and key contributors in specific areas.

Abstract

Glass published the first report on the assessment of systems and software engineering scholars and institutions two decades ago. The ongoing, annual survey of publications in this field provides fund managers, young scholars, graduate students, etc. with useful information for different purposes. However, the studies have been questioned by some critics because of a few shortcomings of the evaluation method. It is actually very hard to reach a widely recognized consensus on such an assessment of scholars and institutions. This paper presents a module and automated method for assessment and trends analysis in software engineering compared with the prior studies. To achieve a more reasonable evaluation result, we take into consideration more high-quality publications, the rank of each publication analyzed, and the different roles of authors named on each paper in question. According to the 7638 papers published in 36 publications from 2008 to 2013, the statistics of research subjects roughly follow power laws, implying the interesting Matthew Effect. We then identify the Top 20 scholars, institutions and countries or regions in terms of a new evaluation rule based on the frequently-used one. The top-ranked scholar is Mark Harman of the University College London, UK, the top-ranked institution is the University of California, USA, and the top-ranked country is the USA. Besides, we also show two levels of trend changes based on the EI classification system and user-defined uncontrolled keywords, as well as noteworthy scholars and institutions in a specific research area. We believe that our results would provide a valuable insight for young scholars and graduate students to seek possible potential collaborators and grasp the popular research topics in software engineering.