hmmSeq: A hidden Markov model for detecting differentially expressed genes from RNA-seq data

TL;DR

hmmSeq is a hierarchical Bayesian hidden Markov model designed to detect differentially expressed genes in RNA-seq data, accounting for gene co-expression, replicates, and paired data, with improved accuracy over existing methods.

Contribution

The paper introduces hmmSeq, a novel hidden Markov model-based approach that incorporates gene co-expression and experimental design features for RNA-seq differential expression analysis.

Findings

Outperforms competitors in ROC curve analysis.

Effectively adjusts for extra-Poisson variability.

Demonstrates power and flexibility on real datasets.

Abstract

We introduce hmmSeq, a model-based hierarchical Bayesian technique for detecting differentially expressed genes from RNA-seq data. Our novel hmmSeq methodology uses hidden Markov models to account for potential co-expression of neighboring genes. In addition, hmmSeq employs an integrated approach to studies with technical or biological replicates, automatically adjusting for any extra-Poisson variability. Moreover, for cases when paired data are available, hmmSeq includes a paired structure between treatments that incoporates subject-specific effects. To perform parameter estimation for the hmmSeq model, we develop an efficient Markov chain Monte Carlo algorithm. Further, we develop a procedure for detection of differentially expressed genes that automatically controls false discovery rate. A simulation study shows that the hmmSeq methodology performs better than competitors in terms of receiver operating characteristic curves. Finally, the analyses of three publicly available RNA-seq data sets demonstrate the power and flexibility of the hmmSeq methodology. An R package implementing the hmmSeq framework will be submitted to CRAN upon publication of the manuscript.