How Data Scientists Review the Scholarly Literature

TL;DR

This study explores how data scientists review scholarly literature, revealing challenges in interdisciplinary understanding, dealing with incomplete details, and leveraging community resources amidst exponential research growth.

Contribution

It provides empirical insights into data scientists' literature review practices and identifies specific challenges and coping strategies in an interdisciplinary, rapidly evolving field.

Findings

Data scientists struggle with interdisciplinary papers and missing details.

They use external resources like code, blogs, and talks to understand research.

Peer support, both online and offline, is crucial for literature review.

Abstract

Keeping up with the research literature plays an important role in the workflow of scientists - allowing them to understand a field, formulate the problems they focus on, and develop the solutions that they contribute, which in turn shape the nature of the discipline. In this paper, we examine the literature review practices of data scientists. Data science represents a field seeing an exponential rise in papers, and increasingly drawing on and being applied in numerous diverse disciplines. Recent efforts have seen the development of several tools intended to help data scientists cope with a deluge of research and coordinated efforts to develop AI tools intended to uncover the research frontier. Despite these trends indicative of the information overload faced by data scientists, no prior work has examined the specific practices and challenges faced by these scientists in an interdisciplinary field with evolving scholarly norms. In this paper, we close this gap through a set of semi-structured interviews and think-aloud protocols of industry and academic data scientists (N = 20). Our results while corroborating other knowledge workers' practices uncover several novel findings: individuals (1) are challenged in seeking and sensemaking of papers beyond their disciplinary bubbles, (2) struggle to understand papers in the face of missing details and mathematical content, (3) grapple with the deluge by leveraging the knowledge context in code, blogs, and talks, and (4) lean on their peers online and in-person. Furthermore, we outline future directions likely to help data scientists cope with the burgeoning research literature.