AutoData: A Multi-Agent System for Open Web Data Collection

TL;DR

AutoData is a multi-agent system that automates web data collection with minimal human input, using a novel architecture and cache system to improve efficiency and reduce costs, validated on new and existing datasets.

Contribution

We introduce AutoData, a multi-agent system with a hypergraph architecture and cache system for scalable, cost-effective web data collection from natural language instructions.

Findings

AutoData outperforms baseline methods on benchmark datasets.

The system effectively reduces token costs in LLM-based data collection.

AutoData successfully collects data across diverse domains like academic, finance, and sports.

Abstract

The exponential growth of data-driven systems and AI technologies has intensified the demand for high-quality web-sourced datasets. While existing datasets have proven valuable, conventional web data collection approaches face significant limitations in terms of human effort and scalability. Current data-collecting solutions fall into two categories: wrapper-based methods that struggle with adaptability and reproducibility, and large language model (LLM)-based approaches that incur substantial computational and financial costs. To address these challenges, we propose AutoData, a novel multi-agent system for Automated web Data collection, that requires minimal human intervention, i.e., only necessitating a natural language instruction specifying the desired dataset. In addition, AutoData is designed with a robust multi-agent architecture, featuring a novel oriented message hypergraph coordinated by a central task manager, to efficiently organize agents across research and development squads. Besides, we introduce a novel hypergraph cache system to advance the multi-agent collaboration process that enables efficient automated data collection and mitigates the token cost issues prevalent in existing LLM-based systems. Moreover, we introduce Instruct2DS, a new benchmark dataset supporting live data collection from web sources across three domains: academic, finance, and sports. Comprehensive evaluations over Instruct2DS and three existing benchmark datasets demonstrate AutoData's superior performance compared to baseline methods. Case studies on challenging tasks such as picture book collection and paper extraction from surveys further validate its applicability. Our source code and dataset are available at https://github.com/GraphResearcher/AutoData.