# Information gains from Monte Carlo Markov Chains

## TL;DR

This paper introduces a new numerical method and Python package for efficiently estimating relative entropy and expected relative entropy from MCMC samples, aiding model comparison and experiment design in cosmology.

## Contribution

The paper presents a novel approach and software tool for computing information-theoretic quantities from MCMC chains, addressing computational challenges in non-Gaussian models.

## Key findings

- Relative error below 0.2% for sample size > 10^5 in Gaussian models
- Method is robust for estimating expected relative entropy
- Provides a practical tool for cosmological data analysis

## Abstract

In this paper, we present a novel method for computing the relative entropy as well as the expected relative entropy using an MCMC chain. The relative entropy from information theory can be used to quantify differences in posterior distributions of a pair of experiments. In cosmology, the relative entropy has been proposed as an interesting tool for model selection, experiment design, forecasting and measuring information gain from subsequent experiments. In contrast to Gaussian distributions, these quantities are not generally available analytically and one needs to use numerical methods to estimate them which are certainly computationally expensive. We propose a method and provide its python package to estimate the relative entropy as well as expected relative entropy from a posterior sample. We consider the linear Gaussian model to check the accuracy of our code. Our results indicate that the relative error is below $0.2\%$ for sample size larger than $10^5$ in the linear Gaussian model. In addition, we study the robustness of our code in estimating the expected relative entropy in this model.