Finite mixtures of matrix-variate Poisson-log normal distributions for three-way count data

TL;DR

This paper introduces a novel mixture model based on matrix variate Poisson-log normal distributions for clustering three-way RNA sequencing count data, effectively capturing complex data structures and reducing parameter complexity.

Contribution

It proposes a new mixture model tailored for three-way count data, with three estimation frameworks and demonstrated effectiveness on real and simulated datasets.

Findings

Successfully recovers underlying cluster structures in data

Shows good parameter recovery in simulations

Effectively models three-way RNA sequencing data

Abstract

Three-way data structures, characterized by three entities, the units, the variables and the occasions, are frequent in biological studies. In RNA sequencing, three-way data structures are obtained when high-throughput transcriptome sequencing data are collected for $n$ genes across $p$ conditions at $r$ occasions. Matrix variate distributions offer a natural way to model three-way data and mixtures of matrix variate distributions can be used to cluster three-way data. Clustering of gene expression data is carried out as means of discovering gene co-expression networks. In this work, a mixture of matrix variate Poisson-log normal distributions is proposed for clustering read counts from RNA sequencing. By considering the matrix variate structure, full information on the conditions and occasions of the RNA sequencing dataset is simultaneously considered, and the number of covariance parameters to be estimated is reduced. We propose three different frameworks for parameter estimation: a Markov chain Monte Carlo based approach, a variational Gaussian approximation based approach, and a hybrid approach. Various information criteria are used for model selection. The models are applied to both real and simulated data, and we demonstrate that the proposed approaches can recover the underlying cluster structure in both cases. In simulation studies where the true model parameters are known, our proposed approach shows good parameter recovery.