Finding Trends in Software Research

TL;DR

This study employs automated text mining with LDADE to analyze 35,391 software engineering papers over 25 years, revealing evolving research topics and demonstrating the importance of automated trend detection in the community.

Contribution

It introduces a fully automated, repeatable method using LDADE for analyzing large-scale research trends in software engineering.

Findings

Identified 11 major research topics in SE.

Showed that research topics evolve over time.

Demonstrated the effectiveness of automated trend detection.

Abstract

This paper explores the structure of research papers in software engineering. Using text mining, we study 35,391 software engineering (SE) papers from 34 leading SE venues over the last 25 years. These venues were divided, nearly evenly, between conferences and journals. An important aspect of this analysis is that it is fully automated and repeatable. To achieve that automation, we used a stable topic modeling technique called LDADE that fully automates parameter tuning in LDA. Using LDADE, we mine 11 topics that represent much of the structure of contemporary SE. The 11 topics presented here should not be "set in stone" as the only topics worthy of study in SE. Rather our goal is to report that (a) text mining methods can detect large scale trends within our community; (b) those topic change with time; so (c) it is important to have automatic agents that can update our understanding of our community whenever new data arrives.