Segmentation of the Poisson and negative binomial rate models: a penalized estimator

TL;DR

This paper introduces a penalized likelihood estimator for segmenting Poisson and negative binomial rate models, effectively determining the number of segments with proven theoretical guarantees and validated on RNA-seq data.

Contribution

It proposes a novel penalized estimator for segmentation of overdispersed count data with non-asymptotic theoretical guarantees.

Findings

Estimator satisfies an oracle inequality.

Performs well on simulated data.

Effective in RNA-seq data analysis.

Abstract

We consider the segmentation problem of Poisson and negative binomial (i.e. overdispersed Poisson) rate distributions. In segmentation, an important issue remains the choice of the number of segments. To this end, we propose a penalized log-likelihood estimator where the penalty function is constructed in a non-asymptotic context following the works of L. Birg\'e and P. Massart. The resulting estimator is proved to satisfy an oracle inequality. The performances of our criterion is assessed using simulated and real datasets in the RNA-seq data analysis context.