Development of Data Evaluation Benchmark for Data Wrangling Recommendation System
Yuqing Wang, Anna Fariha

TL;DR
This paper introduces CoWrangler, a data-wrangling recommendation system, developed through analyzing thousands of Kaggle notebooks to understand common data processing strategies and improve decision-making for data tasks.
Contribution
The paper presents a new benchmark for evaluating data wrangling recommendations based on extensive analysis of real-world user data from Kaggle notebooks.
Findings
Insights into common data processing strategies
Understanding of how dataset quality affects wrangling operations
Potential for expanding dataset sources in future work
Abstract
CoWrangler is a data-wrangling recommender system designed to streamline data processing tasks. Recognizing that data processing is often time-consuming and complex for novice users, we aim to simplify the decision-making process regarding the most effective subsequent data operation. By analyzing over 10,000 Kaggle notebooks spanning approximately 1,000 datasets, we derive insights into common data processing strategies employed by users across various tasks. This analysis helps us understand how dataset quality influences wrangling operations, informing our ongoing efforts to possibly expand our dataset sources in the future.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTechnology and Data Analysis · Innovation in Digital Healthcare Systems
