NoteFlow: Recommending Charts as Sight Glasses for Tracing Data Flow in Computational Notebooks

TL;DR

NoteFlow is a notebook library that recommends adaptive charts as visual aids to help data analysts trace data flow and understand data transformations during exploratory data analysis, reducing cognitive load.

Contribution

It introduces a novel chart recommendation system that adapts to data transformations, enhancing data traceability in computational notebooks.

Findings

User studies show improved understanding of data flow.

Charts effectively convey data transformations.

Enhanced overview of EDA process.

Abstract

Exploratory Data Analysis (EDA) is a routine task for data analysts, often conducted using flexible computational notebooks. During EDA, data workers process, visualize, and interpret data tables, making decisions about subsequent analysis. However, the cell-by-cell programming approach, while flexible, can lead to disorganized code, making it difficult to trace the state of data tables across cells and increasing the cognitive load on data workers. This paper introduces NoteFlow, a notebook library that recommends charts as ``sight glasses'' for data tables, allowing users to monitor their dynamic updates throughout the EDA process. To ensure visual consistency and effectiveness, NoteFlow adapts chart encodings in response to data transformations, maintaining a coherent and insightful representation of the data. The proposed method was evaluated through user studies, demonstrating its ability to provide an overview of the EDA process and convey critical insights in the data tables.