Data Agent: A Holistic Architecture for Orchestrating Data+AI Ecosystems
Zhaoyan Sun, Jiayi Wang, Xinyang Zhao, Jiachi Wang, Guoliang Li

TL;DR
This paper introduces the 'Data Agent', a holistic architecture leveraging large language models to improve orchestration, reasoning, and planning in Data+AI ecosystems, addressing current limitations in semantic understanding and automation.
Contribution
It proposes a comprehensive data agent architecture that integrates LLMs for better understanding, reasoning, and automation in Data+AI system orchestration, with practical examples and open challenges.
Findings
Demonstrates the feasibility of data agents in various Data+AI tasks
Highlights the integration of LLMs enhances system reasoning and planning
Identifies key challenges in designing effective data agent systems
Abstract
Traditional Data+AI systems utilize data-driven techniques to optimize performance, but they rely heavily on human experts to orchestrate system pipelines, enabling them to adapt to changes in data, queries, tasks, and environments. For instance, while there are numerous data science tools available, developing a pipeline planning system to coordinate these tools remains challenging. This difficulty arises because existing Data+AI systems have limited capabilities in semantic understanding, reasoning, and planning. Fortunately, we have witnessed the success of large language models (LLMs) in enhancing semantic understanding, reasoning, and planning abilities. It is crucial to incorporate LLM techniques to revolutionize data systems for orchestrating Data+AI applications effectively. To achieve this, we propose the concept of a 'Data Agent' - a comprehensive architecture designed to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBig Data and Digital Economy · Semantic Web and Ontologies · Multi-Agent Systems and Negotiation
