Inferring the shape of data: A probabilistic framework for analyzing experiments in the natural sciences

TL;DR

This paper introduces a Bayesian probabilistic framework for objectively determining whether data conforms to expected shapes, improving feature detection and comparison in high-dimensional scientific datasets.

Contribution

It presents a novel Bayesian method for shape inference in datasets, enabling automated, quantitative analysis of features in complex scientific data.

Findings

Demonstrates the framework with proof-of-principle examples

Enables objective comparison of theoretical models with data

Automates feature detection in high-dimensional datasets

Abstract

A critical step in data analysis for many different types of experiments is the identification of features with theoretically defined shapes in N-dimensional datasets; examples of this process include finding peaks in multi-dimensional molecular spectra or emitters in fluorescence microscopy images. Identifying such features involves determining if the overall shape of the data is consistent with an expected shape, however, it is generally unclear how to quantitatively make this determination. In practice, many analysis methods employ subjective, heuristic approaches, which complicates the validation of any ensuing results - especially as the amount and dimensionality of the data increase. Here, we present a probabilistic solution to this problem by using Bayes' rule to calculate the probability that the data has any one of several potential shapes. This probabilistic approach may be used to objectively compare how well different theories describe a dataset, identify changes between datasets, and detect features within data using a corollary method called Bayesian Inference-based Template Search (BITS); several proof-of-principle examples are provided. Altogether, this mathematical framework serves as an automated 'engine' capable of computationally executing analysis decisions currently made by visual inspection across the sciences.