SketchFill: Sketch-Guided Code Generation for Imputing Derived Missing Values

TL;DR

SketchFill introduces a sketch-guided approach to enhance large language models in imputing missing derived numerical values in data, significantly improving accuracy over existing methods.

Contribution

The paper presents SketchFill, a novel sketch-based technique that guides LLMs to generate formulas for imputing missing data, addressing limitations of prior methods in complex reasoning tasks.

Findings

Achieves 56.2% higher accuracy than Chain-of-Thought methods

Achieves 78.8% higher accuracy than MetaGPT

Sets a new standard for automated data cleaning

Abstract

Missing value is a critical issue in data science, significantly impacting the reliability of analyses and predictions. Missing value imputation (MVI) is a longstanding problem because it highly relies on domain knowledge. Large language models (LLMs) have emerged as a promising tool for data cleaning, including MVI for tabular data, offering advanced capabilities for understanding and generating content. However, despite their promise, existing LLM techniques such as in-context learning and Chain-of-Thought (CoT) often fall short in guiding LLMs to perform complex reasoning for MVI, particularly when imputing derived missing values, which require mathematical formulas and data relationships across rows and columns. This gap underscores the need for further advancements in LLM methodologies to enhance their reasoning capabilities for more reliable imputation outcomes. To fill this gap, we propose SketchFill, a novel sketch-based method to guide LLMs in generating accurate formulas to impute missing numerical values. Our experimental results demonstrate that SketchFill significantly outperforms state-of-the-art methods, achieving 56.2% higher accuracy than CoT-based methods and 78.8% higher accuracy than MetaGPT. This sets a new standard for automated data cleaning and advances the field of MVI for numerical values.