# Differentially expressed heterogeneous overdispersion genes testing for count data

## TL;DR

This paper introduces a new method for identifying differentially expressed genes in RNA-seq data that improves detection power when sample sizes are small.

## Contribution

The novel DEHOGT method uses heterogeneous overdispersion modeling to better detect differentially expressed genes in RNA-seq data.

## Key findings

- DEHOGT outperforms DESeq2 and EdgeR in detecting differentially expressed genes in synthetic RNA-seq data.
- DEHOGT detects more differentially expressed genes in microglial cells under stress hormone treatments.
- The method enhances detection power when the number of replicates is limited but the number of conditions is large.

## Abstract

The mRNA-seq data analysis is a powerful technology for inferring information from biological systems of interest. Specifically, the sequenced RNA fragments are aligned with genomic reference sequences, and we count the number of sequence fragments corresponding to each gene for each condition. A gene is identified as differentially expressed (DE) if the difference in its count numbers between conditions is statistically significant. Several statistical analysis methods have been developed to detect DE genes based on RNA-seq data. However, the existing methods could suffer decreasing power to identify DE genes arising from overdispersion and limited sample size, where overdispersion refers to the empirical phenomenon that the variance of read counts is larger than the mean of read counts. We propose a new differential expression analysis procedure: heterogeneous overdispersion genes testing (DEHOGT) based on heterogeneous overdispersion modeling and a post-hoc inference procedure. DEHOGT integrates sample information from all conditions and provides a more flexible and adaptive overdispersion modeling for the RNA-seq read count. DEHOGT adopts a gene-wise estimation scheme to enhance the detection power of differentially expressed genes when the number of replicates is limited as long as the number of conditions is large. DEHOGT is tested on the synthetic RNA-seq read count data and outperforms two popular existing methods, DESeq2 and EdgeR, in detecting DE genes. We apply the proposed method to a test dataset using RNAseq data from microglial cells. DEHOGT tends to detect more differently expressed genes potentially related to microglial cells under different stress hormones treatments.