Modelling overdispersion heterogeneity in differential expression analysis using mixtures

TL;DR

This paper introduces a mixture model approach for more reliable overdispersion estimation in RNA-seq differential expression analysis, improving sensitivity and controlling false positives.

Contribution

It proposes a novel mixture model that shares information across genes to better estimate overdispersion, addressing limitations of existing plug-in methods.

Findings

Improved detection sensitivity for differentially expressed genes.

Better control of type-I error rates.

Validated on prostate cancer RNA-seq data.

Abstract

Next-generation sequencing technologies now constitute a method of choice to measure gene expression. Data to analyze are read counts, commonly modeled using Negative Binomial distributions. A relevant issue associated with this probabilistic framework is the reliable estimation of the overdispersion parameter, reinforced by the limited number of replicates generally observable for each gene. Many strategies have been proposed to estimate this parameter, but when differential analysis is the purpose, they often result in procedures based on plug-in estimates, and we show here that this discrepancy between the estimation framework and the testing framework can lead to uncontrolled type-I errors. Instead we propose a mixture model that allows each gene to share information with other genes that exhibit similar variability. Three consistent statistical tests are developed for differential expression analysis. We show that the proposed method improves the sensitivity of detecting differentially expressed genes with respect to the common procedures, since it is the best one in reaching the nominal value for the first-type error, while keeping elevate power. The method is finally illustrated on prostate cancer RNA-seq data.