Dark Sky Simulations: Early Data Release

TL;DR

The Dark Sky Simulations provide extensive, high-resolution cosmological data to model the universe's evolution, including innovative data distribution methods for large datasets, aiding dark matter and dark energy research.

Contribution

This paper introduces a large-scale cosmological simulation dataset and a novel web-based method for accessing and analyzing massive distributed datasets.

Findings

Mass function and power spectrum are consistent within 1% over a wide range.

Over 55 Terabytes of simulation data released to the community.

New data access method enables efficient querying and analysis of large datasets.

Abstract

The Dark Sky Simulations are an ongoing series of cosmological N-body simulations designed to provide a quantitative and accessible model of the evolution of the large-scale Universe. Such models are essential for many aspects of the study of dark matter and dark energy, since we lack a sufficiently accurate analytic model of non-linear gravitational clustering. In July 2014, we made available to the general community our early data release, consisting of over 55 Terabytes of simulation data products, including our largest simulation to date, which used $1.07 \times 10^{12}~(10240^3)$ particles in a volume $8h^{-1}\mathrm{Gpc}$ across. Our simulations were performed with 2HOT, a purely tree-based adaptive N-body method, running on 200,000 processors of the Titan supercomputer, with data analysis enabled by yt. We provide an overview of the derived halo catalogs, mass function, power spectra and light cone data. We show self-consistency in the mass function and mass power spectrum at the 1% level over a range of more than 1000 in particle mass. We also present a novel method to distribute and access very large datasets, based on an abstraction of the World Wide Web (WWW) as a file system, remote memory-mapped file access semantics, and a space-filling curve index. This method has been implemented for our data release, and provides a means to not only query stored results such as halo catalogs, but also to design and deploy new analysis techniques on large distributed datasets.