SCISPACE: A Scientific Collaboration Workspace for File Systems in Geo-Distributed HPC Data Centers

TL;DR

SCISPACE is a collaborative workspace system designed for geo-distributed HPC data centers, enabling high-performance native data access, efficient search, and improved scientific collaboration over terabit networks.

Contribution

It introduces a novel workspace system that integrates native data access and search capabilities for geo-distributed HPC data centers, enhancing collaboration efficiency.

Findings

Achieved 36% performance boost with native data access.

Demonstrated feasibility using real scientific datasets.

Supported high-performance collaboration across geo-distributed data centers.

Abstract

Future terabit networks are committed to dramatically improving big data motion between geographically dispersed HPC data centers.The scientific community takes advantage of the terabit networks such as DOE's ESnet and accelerates the trend to build a small world of collaboration between geospatial HPC data centers. It improves information and resource sharing for joint simulation and analysis between the HPC data centers. In this paper, we propose to build SCISPACE (Scientific Collaboration Workspace) for collaborative data centers. It provides a global view of information shared from multiple geo-distributed HPC data centers under a single workspace. SCISPACE supports native data-access to gain high-performance when data read or write is required in native data center namespace. It is accomplished by integrating a metadata export protocol. To optimize scientific collaborations across HPC data centers, SCISPACE implements search and discovery service. To evaluate, we configured two geo-distributed small-scale HPC data centers connected via high-speed Infiniband network, equipped with LustreFS. We show the feasibility of SCISPACE using real scientific datasets and applications. The evaluation results show average 36\% performance boost when the proposed native-data access is employed in collaborations.