When Are Reactive Notebooks Not Reactive?
Megan Zheng, Will Crichton, Akshay Narayan, Deepti Raghavan, Nikos Vasilakis

TL;DR
This paper introduces Rex, a test suite to evaluate reactivity in notebook systems, revealing inconsistencies and failures that hinder user understanding and system reliability.
Contribution
It provides a systematic way to assess reactivity in notebooks, highlighting issues and guiding improvements in reactive notebook implementations.
Findings
Rex identifies various failure modes in existing reactive notebooks.
Different systems define reactivity in inconsistent ways.
Reactivity failures can break user mental models.
Abstract
Computational notebooks are convenient for programmers, but can easily become confusing and inconsistent due to the ability to incrementally edit a program that is running. Recent reactive notebook systems, such as Ipyflow, Marimo and Observable, strive to keep notebook state in sync with the current cell code by re-executing a minimal set of cells upon modification. However, each system defines reactivity a different way. Additionally, within any definition, we find simple notebook modifications that can break each system. Overall, these inconsistencies make it difficult for users to construct a mental model of their reactive notebook's implementation. This paper proposes Rex, a fine-grained test suite to discuss and assess reactivity capabilities within reactive notebook systems. We evaluate Rex on three existing reactive notebook systems and classify their failures with the aims of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Software Testing and Debugging Techniques · Logic, programming, and type systems
