Observing Fine-Grained Changes in Jupyter Notebooks During Development Time

TL;DR

This paper introduces a toolset for tracking and analyzing fine-grained changes in Jupyter notebooks during development, providing new insights into the dynamic nature of notebook-based data science workflows.

Contribution

It presents a novel toolset and dataset for observing code changes in Jupyter notebooks, filling a research gap in computational notebook analysis.

Findings

Many changes are code iteration modifications

Significant dynamic activity occurs during notebook development

Dataset includes over 2,600 cells and 9,200 executions

Abstract

In software engineering, numerous studies have focused on the analysis of fine-grained logs, leading to significant innovations in areas such as refactoring, security, and code completion. However, no similar studies have been conducted for computational notebooks in the context of data science. To help bridge this research gap, we make three scientific contributions: we (1) introduce a toolset for collecting code changes in Jupyter notebooks during development time; (2) use it to collect more than 100 hours of work related to a data analysis task and a machine learning task (carried out by 20 developers with different levels of expertise), resulting in a dataset containing 2,655 cells and 9,207 cell executions; and (3) use this dataset to investigate the dynamic nature of the notebook development process and the changes that take place in the notebooks. In our analysis of the collected data, we classified the changes made to the cells between executions and found that a significant number of these changes were code iteration modifications. We report a number of other insights and propose potential future research directions on the novel data.