Any Data, Any Time, Anywhere: Global Data Access for Science

TL;DR

The paper introduces the AAA infrastructure, a unified global data access system that simplifies data retrieval across multiple storage sites for scientific research, demonstrated through its application in high-energy physics experiments.

Contribution

It presents a novel integration of existing software to create a global data federation, enabling seamless data access for distributed high-throughput computing in science.

Findings

Successful implementation in CMS experiment

Improved data access efficiency

Unified view of distributed storage systems

Abstract

Data access is key to science driven by distributed high-throughput computing (DHTC), an essential technology for many major research projects such as High Energy Physics (HEP) experiments. However, achieving efficient data access becomes quite difficult when many independent storage sites are involved because users are burdened with learning the intricacies of accessing each system and keeping careful track of data location. We present an alternate approach: the Any Data, Any Time, Anywhere infrastructure. Combining several existing software products, AAA presents a global, unified view of storage systems - a "data federation," a global filesystem for software delivery, and a workflow management system. We present how one HEP experiment, the Compact Muon Solenoid (CMS), is utilizing the AAA infrastructure and some simple performance metrics.