Bridging Simulations and Observations: New Insights into Galaxy Formation Simulations via Out-of-Distribution Detection and Bayesian Model Comparison

TL;DR

This paper introduces a novel framework combining Bayesian model comparison and out-of-distribution detection to evaluate the realism of galaxy formation simulations against observational data, addressing the challenge of high-dimensional data and limited simulation budgets.

Contribution

It develops a new method integrating VAE-based embeddings, OOD detection, and Bayesian comparison to assess and compare galaxy simulations with real observations.

Findings

Identifies gaps between simulations and real galaxy images.

Determines the best-performing simulation model among six tested.

Provides partial explanations for model performance using SHAP values.

Abstract

Cosmological simulations are a powerful tool to advance our understanding of galaxy formation and many simulations model key properties of real galaxies. A question that naturally arises for such simulations in light of high-quality observational data is: How close are the models to reality? Due to the high-dimensionality of the problem, many previous studies evaluate galaxy simulations using simplified summary statistics of physical properties. In this work, we combine simulation-based Bayesian model comparison with a novel misspecification detection technique to compare simulated galaxy images of 6 hydrodynamical models observations. Since cosmological simulations are computationally costly, we address the problem of low simulation budgets by first training a $k$-sparse variational autoencoder (VAE) on the abundant dataset of SDSS images. The VAE learns to extract informative latent embeddings and delineates the typical set of real images. To reveal simulation gaps, we then perform out-of-distribution detection (OOD) based on the logits of classifiers trained on the embeddings of simulated images. Finally, we perform amortized Bayesian model comparison using probabilistic classification, identifying the relatively best-performing model along with partial explanations through SHAP values.