Untangling Knots: Leveraging LLM for Error Resolution in Computational Notebooks

TL;DR

This paper explores using large language models to improve error resolution in computational notebooks, addressing challenges of reproducibility and bugs with a novel dataset and an iterative agent approach.

Contribution

It introduces an LLM-based iterative agent for bug fixing in notebooks and provides a new dataset to support research in this area.

Findings

Proposed an LLM-driven approach for notebook error resolution.

Created a dataset of computational notebooks with bugs.

Discussed challenges and potential of LLMs in this context.

Abstract

Computational notebooks became indispensable tools for research-related development, offering unprecedented interactivity and flexibility in the development process. However, these benefits come at the cost of reproducibility and an increased potential for bugs. There are many tools for bug fixing; however, they are generally targeted at the classical linear code. With the rise of code-fluent Large Language Models, a new stream of smart bug-fixing tools has emerged. However, the applicability of those tools is still problematic for non-linear computational notebooks. In this paper, we propose a potential solution for resolving errors in computational notebooks via an iterative LLM-based agent. We discuss the questions raised by this approach and share a novel dataset of computational notebooks containing bugs to facilitate the research of the proposed approach.