EVEREST: A design environment for extreme-scale big data analytics on heterogeneous platforms

TL;DR

EVEREST is a comprehensive design environment aimed at optimizing high-performance big data analytics on heterogeneous platforms by integrating advanced programming models, hardware-accelerated AI, and runtime monitoring.

Contribution

It introduces a holistic environment for co-designing HPDA applications on diverse, distributed, and secure hardware platforms, emphasizing programmability and resource efficiency.

Findings

Development of a data-driven design approach

Integration of hardware-accelerated AI techniques

Implementation of efficient runtime monitoring with virtualization

Abstract

High-Performance Big Data Analytics (HPDA) applications are characterized by huge volumes of distributed and heterogeneous data that require efficient computation for knowledge extraction and decision making. Designers are moving towards a tight integration of computing systems combining HPC, Cloud, and IoT solutions with artificial intelligence (AI). Matching the application and data requirements with the characteristics of the underlying hardware is a key element to improve the predictions thanks to high performance and better use of resources. We present EVEREST, a novel H2020 project started on October 1st, 2020 that aims at developing a holistic environment for the co-design of HPDA applications on heterogeneous, distributed, and secure platforms. EVEREST focuses on programmability issues through a data-driven design approach, the use of hardware-accelerated AI, and an efficient runtime monitoring with virtualization support. In the different stages, EVEREST combines state-of-the-art programming models, emerging communication standards, and novel domain-specific extensions. We describe the EVEREST approach and the use cases that drive our research.