An Hierarchical Approach to Big Data

TL;DR

This paper presents the Hierarchical Progressive Survey (HiPS), a multi-resolution scheme based on HEALPix for efficient exploration, access, and visualization of large, heterogeneous astronomical datasets, facilitating collaboration and data management.

Contribution

It introduces a practical hierarchical approach for managing big astronomical data using HiPS, enabling multi-resolution visualization and easy implementation for diverse datasets.

Findings

Over 250 diverse datasets available via HiPS network

Multiple mirror implementations enhance data accessibility

Ease of implementation promotes widespread adoption

Abstract

The increasing volumes of astronomical data require practical methods for data exploration, access and visualisation. The Hierarchical Progressive Survey (HiPS) is a HEALPix based scheme that enables a multi-resolution approach to astronomy data from the individual pixels up to the whole sky. We highlight the decisions and approaches that have been taken to make this scheme a practical solution for managing large volumes of heterogeneous data. Early implementors of this system have formed a network of HiPS nodes, with some 250 diverse data sets currently available, with multiple mirror implementations for important data sets. This hierarchical approach can be adapted to expose Big Data in different ways. We describe how the ease of implementation, and local customisation of the Aladin Lite embeddable HiPS visualiser have been keys for promoting collaboration on HiPS.