AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science

TL;DR

This paper introduces AssistedDS, a benchmark to evaluate how well large language models can utilize external domain knowledge in automated data science tasks, revealing significant limitations in their critical evaluation abilities.

Contribution

The paper presents AssistedDS, a novel benchmark with synthetic and real datasets to systematically assess LLMs' handling of domain knowledge in data science workflows.

Findings

LLMs often uncritically adopt provided information, harming performance with adversarial content.

Helpful guidance alone cannot prevent negative effects of adversarial data.

LLMs struggle with time-series data, feature engineering, and categorical variables in Kaggle datasets.

Abstract

Large language models (LLMs) have advanced the automation of data science workflows. Yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice. To answer this question, we introduce AssistedDS (Assisted Data Science), a benchmark designed to systematically evaluate how LLMs handle domain knowledge in tabular prediction tasks. AssistedDS features both synthetic datasets with explicitly known generative mechanisms and real-world Kaggle competitions, each accompanied by curated bundles of helpful and adversarial documents. These documents provide domain-specific insights into data cleaning, feature engineering, and model selection. We assess state-of-the-art LLMs on their ability to discern and apply beneficial versus harmful domain knowledge, evaluating submission validity, information recall, and predictive performance. Our results demonstrate three key findings: (1) LLMs frequently exhibit an uncritical adoption of provided information, significantly impairing their predictive performance when adversarial content is introduced, (2) helpful guidance is often insufficient to counteract the negative influence of adversarial information, and (3) in Kaggle datasets, LLMs often make errors in handling time-series data, applying consistent feature engineering across different folds, and interpreting categorical variables correctly. These findings highlight a substantial gap in current models' ability to critically evaluate and leverage expert knowledge, underscoring an essential research direction for developing more robust, knowledge-aware automated data science systems. Our data and code are publicly available here: https://github.com/jeremyxianx/Assisted-DS