On the Verge of One Petabyte - the Story Behind the BaBar Database System

TL;DR

The BaBar database system, utilizing Objectivity/DB, has successfully managed over 700 terabytes of data, demonstrating scalable architecture and ongoing improvements for large-scale scientific data management.

Contribution

This paper details the development and experience of managing a large-scale, object-oriented database system in high-energy physics, highlighting universal strategies applicable to other large systems.

Findings

Managed over 700 terabytes of data since 1999

Implemented features to reduce space and operational costs

Provided insights into large-scale database system development

Abstract

The BaBar database has pioneered the use of a commercial ODBMS within the HEP community. The unique object-oriented architecture of Objectivity/DB has made it possible to manage over 700 terabytes of production data generated since May'99, making the BaBar database the world's largest known database. The ongoing development includes new features, addressing the ever-increasing luminosity of the detector as well as other changing physics requirements. Significant efforts are focused on reducing space requirements and operational costs. The paper discusses our experience with developing a large scale database system, emphasizing universal aspects which may be applied to any large scale system, independently of underlying technology used.