The SAGE Project: a Storage Centric Approach for Exascale Computing

TL;DR

The SAGE project introduces a storage-centric infrastructure for exascale computing that integrates diverse storage technologies and software to handle massive data volumes for scientific analysis and simulations.

Contribution

It presents the design, implementation, and software architecture of a storage-centric exascale system, including a prototype at Julich Supercomputing Center.

Findings

Prototype system successfully implemented and installed.

Multi-tier storage hierarchy with diverse technologies.

Software architecture supports large-scale data processing.

Abstract

SAGE (Percipient StorAGe for Exascale Data Centric Computing) is a European Commission funded project towards the era of Exascale computing. Its goal is to design and implement a Big Data/Extreme Computing (BDEC) capable infrastructure with associated software stack. The SAGE system follows a "storage centric" approach as it is capable of storing and processing large data volumes at the Exascale regime. SAGE addresses the convergence of Big Data Analysis and HPC in an era of next-generation data centric computing. This convergence is driven by the proliferation of massive data sources, such as large, dispersed scientific instruments and sensors where data needs to be processed, analyzed and integrated into simulations to derive scientific and innovative insights. A first prototype of the SAGE system has been been implemented and installed at the Julich Supercomputing Center. The SAGE storage system consists of multiple types of storage device technologies in a multi-tier I/O hierarchy, including flash, disk, and non-volatile memory technologies. The main SAGE software component is the Seagate Mero Object Storage that is accessible via the Clovis API and higher level interfaces. The SAGE project also includes scientific applications for the validation of the SAGE concepts. The objective of this paper is to present the SAGE project concepts, the prototype of the SAGE platform and discuss the software architecture of the SAGE system.