A Big Data Analyzer for Large Trace Logs
Alkida Balliu, Dennis Olivetti, Ozalp Babaoglu, Moreno Marzolla, Alina, S\^irbu

TL;DR
BiDAl is a versatile Java-based tool integrating Big Data technologies like SQL, R, and Hadoop to analyze large-scale data center logs, aiding in pattern discovery and system modeling.
Contribution
The paper introduces BiDAl, a modular, multi-language analysis platform that simplifies large trace log analysis for data centers using integrated Big Data tools.
Findings
Successfully analyzed Google data cluster traces
Built realistic data center models from logs
Demonstrated extensibility of BiDAl platform
Abstract
Current generation of Internet-based services are typically hosted on large data centers that take the form of warehouse-size structures housing tens of thousands of servers. Continued availability of a modern data center is the result of a complex orchestration among many internal and external actors including computing hardware, multiple layers of intricate software, networking and storage devices, electrical power and cooling plants. During the course of their operation, many of these components produce large amounts of data in the form of event and error logs that are essential not only for identifying and resolving problems but also for improving data center efficiency and management. Most of these activities would benefit significantly from data analytics techniques to exploit hidden statistical patterns and correlations that may be present in the data. The sheer volume of data to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
