Assessing, testing and estimating the amount of fine-tuning by means of active information

TL;DR

This paper introduces a framework to quantify how much external, pre-specified knowledge influences search algorithms, using active information and fine-tuning tests, with applications across diverse scientific fields.

Contribution

It develops a general method to estimate active information and fine-tuning in algorithms, including parametric and nonparametric tests, and applies Markov chain models for analysis.

Findings

Framework for estimating active information in search algorithms

Method for testing fine-tuning using exponential tilting and Markov chains

Applications demonstrated in cosmology, genetics, and reinforcement learning

Abstract

A general framework is introduced to estimate how much external information has been infused into a search algorithm, the so-called active information. This is rephrased as a test of fine-tuning, where tuning corresponds to the amount of pre-specified knowledge that the algorithm makes use of in order to reach a certain target. A function $f$ quantifies specificity for each possible outcome $x$ of a search, so that the target of the algorithm is a set of highly specified states, whereas fine-tuning occurs if it is much more likely for the algorithm to reach the target than by chance. The distribution of a random outcome $X$ of the algorithm involves a parameter $\theta$ that quantifies how much background information that has been infused. A simple choice of this parameter is to use $\theta f$ in order to exponentially tilt the distribution of the outcome of the search algorithm under the null distribution of no tuning, so that an exponential family of distributions is obtained. Such algorithms are obtained by iterating a Metropolis-Hastings type of Markov chain, and this makes it possible to compute the their active information under equilibrium and non-equilibrium of the Markov chain, with or without stopping when the targeted set of fine-tuned states has been reached. Other choices of tuning parameters $\theta$ are discussed as well. Nonparametric and parametric estimators of active information and tests of fine-tuning are developed when repeated and independent outcomes of the algorithm are available. The theory is illustrated with examples from cosmology, student learning, reinforcement learning, a Moran type model of population genetics, and evolutionary programming.