DataParasite Enables Scalable and Repurposable Online Data Curation
Mengyi Sun (Cold Spring Harbor Laboratory)

TL;DR
DataParasite is an open-source, modular pipeline that enables scalable, accurate, and cost-effective online data collection for social science research, adaptable to various tasks with minimal manual effort.
Contribution
It introduces a flexible, reusable data curation pipeline that decomposes tasks into entity-level searches, usable across multiple domains with natural-language instructions.
Findings
Achieves high accuracy across diverse social science datasets.
Reduces data collection costs by an order of magnitude.
Demonstrates versatility in multiple canonical tasks.
Abstract
Many questions in computational social science rely on datasets assembled from heterogeneous online sources, a process that is often labor-intensive, costly, and difficult to reproduce. Recent advances in large language models enable agentic search and structured extraction from the web, but existing systems are frequently opaque, inflexible, or poorly suited to scientific data curation. Here we introduce DataParasite, an open-source, modular pipeline for scalable online data collection. DataParasite decomposes tabular curation tasks into independent, entity-level searches defined through lightweight configuration files and executed through a shared, task-agnostic python script. Crucially, the same pipeline can be repurposed to new tasks, including those without predefined entity lists, using only natural-language instructions. We evaluate the pipeline on multiple canonical tasks in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsScientific Computing and Data Management · Computational and Text Analysis Methods · Research Data Management Practices
