Impact of Software Engineering Research in Practice: A Patent and Author Survey Analysis

TL;DR

This study analyzes how software engineering research has been practically applied through patents and author surveys, revealing the impact, challenges, and the importance of cross-disciplinary and practitioner-oriented research.

Contribution

It provides a comprehensive quantitative and qualitative analysis of SE research impact via patents and author surveys, highlighting practical applications and influencing factors.

Findings

Researchers have provided tools, processes, and methods to practitioners.

Practitioner-oriented venues are more impactful than researcher-oriented ones.

Impact is influenced by funding, cost-benefit trade-offs, and cross-disciplinary relevance.

Abstract

Existing work on the practical impact of software engineering (SE) research examines industrial relevance rather than adoption of study results, hence the question of how results have been practically applied remains open. To answer this and investigate the outcomes of impactful research, we performed a quantitative and qualitative analysis of 4,354 SE patents citing 1,690 SE papers published in four leading SE venues between 1975-2017. Moreover, we conducted a survey on 475 authors of 593 top-cited and awarded publications, achieving 26% response rate. Overall, researchers have equipped practitioners with various tools, processes, and methods, and improved many existing products. SE practice values knowledge-seeking research and is impacted by diverse cross-disciplinary SE areas. Practitioner-oriented publication venues appear more impactful than researcher-oriented ones, while industry-related tracks in conferences could enhance their impact. Some research works did not reach a wide footprint due to limited funding resources or unfavorable cost-benefit trade-off of the proposed solutions. The need for higher SE research funding could be corroborated through a dedicated empirical study. In general, the assessment of impact is subject to its definition. Therefore, academia and industry could jointly agree on a formal description to set a common ground for subsequent research on the topic.