Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks

TL;DR

This paper introduces a James-Stein-type shrinkage estimator for entropy and mutual information, enabling effective inference of gene association networks from small-sample high-dimensional data, outperforming existing methods.

Contribution

It develops a novel shrinkage estimator for entropy and mutual information, improving accuracy and efficiency in small-sample, high-dimensional settings, especially for gene network inference.

Findings

Outperforms eight other entropy estimators across various scenarios

Efficiently infers gene association networks from limited data

Provides a publicly available implementation of the estimator

Abstract

We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically as well as computationally. Despite its simplicity, we show that it outperforms eight other entropy estimation procedures across a diverse range of sampling scenarios and data-generating models, even in cases of severe undersampling. We illustrate the approach by analyzing E. coli gene expression data and computing an entropy-based gene-association network from gene expression data. A computer program is available that implements the proposed shrinkage estimator.