Toward Living Narrative Reviews: An Empirical Study of the Processes and Challenges in Updating Survey Articles in Computing Research

TL;DR

This study investigates the challenges and processes involved in updating survey articles in computing research, highlighting the need for better tools and incentives to keep reviews current and accurate.

Contribution

It provides an empirical analysis of researchers' practices and barriers in updating survey articles, offering insights for designing supportive technologies.

Findings

Researchers value updated surveys but find continuous updating unmanageable.

Current workflows are misaligned with academic incentives for updating.

Opportunities exist for technological solutions to facilitate survey updates.

Abstract

Surveying prior literature to establish a foundation for new knowledge is essential for scholarly progress. However, survey articles are resource-intensive and challenging to create, and can quickly become outdated as new research is published, risking information staleness and inaccuracy. Keeping survey articles current with the latest evidence is therefore desirable, though there is a limited understanding of why, when, and how these surveys should be updated. Toward this end, through a series of in-depth retrospective interviews with 11 researchers, we present an empirical examination of the work practices in authoring and updating survey articles in computing research. We find that while computing researchers acknowledge the value in maintaining an updated survey, continuous updating remains unmanageable and misaligned with academic incentives. Our findings suggest key leverage points within current workflows that present opportunities for enabling technologies to facilitate more efficient and effective updates.