Limit Theorems for Empirical R\'enyi Entropy and Divergence with Applications to Molecular Diversity Analysis

TL;DR

This paper develops limit theorems for empirical Renyi entropy and divergence in large, sparse, and unbalanced datasets, enabling more accurate molecular biodiversity analysis from high-throughput sequencing data.

Contribution

It introduces a frequency-based framework with Gaussian limit results for Renyi entropy and divergence applicable to large, sparse, and heavy-tailed genomic data.

Findings

Validates methods with RNA sequencing examples

Addresses challenges of large, unbalanced contingency tables

Provides theoretical foundation for molecular diversity metrics

Abstract

Quantitative methods for studying biodiversity have been traditionally rooted in the classical theory of finite frequency tables analysis. However, with the help of modern experimental tools, like high throughput sequencing, we now begin to unlock the outstanding diversity of genomic data in plants and animals reflective of the long evolutionary history of our planet. This molecular data often defies the classical frequency/contingency tables assumptions and seems to require sparse tables with very large number of categories and highly unbalanced cell counts, e.g., following heavy tailed distributions (for instance, power laws). Motivated by the molecular diversity studies, we propose here a frequency-based framework for biodiversity analysis in the asymptotic regime where the number of categories grows with sample size (an infinite contingency table). Our approach is rooted in information theory and based on the Gaussian limit results for the effective number of species (the Hill numbers) and the empirical Renyi entropy and divergence. We argue that when applied to molecular biodiversity analysis our methods can properly account for the complicated data frequency patterns on one hand and the practical sample size limitations on the other. We illustrate this principle with two specific RNA sequencing examples: a comparative study of T-cell receptor populations and a validation of some preselected molecular hepatocellular carcinoma (HCC) markers.