Basic Experiment Planning via Information Metrics: the RoboMendel Problem

TL;DR

This paper develops an information-theoretic framework for experiment planning, aiming to mathematically optimize the selection of experiments to efficiently uncover scientific principles, exemplified through a genetics case study.

Contribution

It introduces a rigorous information metric for experiment selection that converges to mutual information, enabling optimal experiment sequencing in scientific discovery.

Findings

The information metric can predict the most informative experiments.

The approach computes the potential information yield of experiments.

Applied to genetics, it effectively guides experiment sequences.

Abstract

In this paper we outline some mathematical questions that emerge from trying to "turn the scientific method into math". Specifically, we consider the problem of experiment planning (choosing the best experiment to do next) in explicit probabilistic and information theoretic terms. We formulate this as an information measurement problem; that is, we seek a rigorous definition of an information metric to measure the likely information yield of an experiment, such that maximizing the information metric will indeed reliably choose the best experiment to perform. We present the surprising result that defining the metric purely in terms of prediction power on observable variables yields a metric that can converge to the classical mutual information measuring how informative the experimental observation is about an underlying hidden variable. We show how the expectation potential information metric can compute the "information rate" of an experiment as well its total possible yield, and the information value of experimental controls. To illustrate the utility of these concepts for guiding fundamental scientific inquiry, we present an extensive case study (RoboMendel) applying these metrics to propose sequences of experiments for discovering the basic principles of genetics.