Exascale In-situ visualization for Astronomy & Cosmology

TL;DR

This paper presents an in-situ visualization framework for Astronomy & Cosmology simulations, enabling real-time analysis of large data volumes directly during simulations, thus overcoming storage bottlenecks in exascale computing.

Contribution

The work introduces Hecuba integrated with ChaNGa to facilitate real-time, in-situ visualization of cosmological simulation data, a novel approach in this domain.

Findings

Enables real-time visualization of large-scale cosmological simulations.

Reduces data storage and I/O bottlenecks in high-performance computing.

Supports integration with visualization tools like ParaView and VisIVO.

Abstract

Modern simulations and observations in Astronomy & Cosmology (A&C) produce massively large data volumes, posing significant challenges for storage, access and data analysis. A long-standing bottleneck in high-performance computing, especially now in the exascale era, has been the requirement to write these large datasets to disks, which limits the performance. A promising solution to this challenge is in-situ processing, where analysis and visualization are performed concurrently with the simulation itself, bypassing the storage of the simulation data. In this work, we present new results from an approach for in-situ processing based on Hecuba, a framework that provides a highly distributed database for streaming A&C simulation data directly into the visualization pipeline to make possible on-line visualization. By integrating Hecuba with the high-performance cosmological simulator ChaNGa, we enable real-time, in-situ visualization of N-body simulation results using tools such as ParaView and VisIVO.