biggy: An Implementation of Unified Framework for Big Data Management System
Yao Wu, Henan Guan

TL;DR
This paper introduces Biggy, an implementation of the Datar framework, a unified, pluggable, and intelligent system architecture for managing big data efficiently across various functionalities.
Contribution
It proposes the Datar framework for big data management, integrating system architecture ideas with features like pluggable engines and automatic dataflow, and demonstrates Biggy as its implementation.
Findings
Biggy shows improved efficiency and scalability in big data tasks.
The framework enables automatic and intelligent management of big data workflows.
Datar's architecture supports flexible and modular big data system design.
Abstract
Various tools, softwares and systems are proposed and implemented to tackle the challenges in big data on different emphases, e.g., data analysis, data transaction, data query, data storage, data visualization, data privacy. In this paper, we propose datar, a new prospective and unified framework for Big Data Management System (BDMS) from the point of system architecture by leveraging ideas from mainstream computer structure. We introduce five key components of datar by reviewing the cur- rent status of BDMS. Datar features with configuration chain of pluggable engines, automatic dataflow on job pipelines, intelligent self-driving system management and interactive user interfaces. Moreover, we present biggy as an implementation of datar with manipulation details demonstrated by four running examples. Evaluations on efficiency and scalability are carried out to show the performance. Our…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCloud Computing and Resource Management · Big Data Technologies and Applications · Big Data and Business Intelligence
