A Family of Mixture Models for Biclustering

TL;DR

This paper introduces a flexible family of biclustering models based on mixture models with diagonal covariance matrices, improving fit on complex data by allowing non-zero off-diagonal elements.

Contribution

It proposes a new class of biclustering models that relax previous restrictions, using diagonal covariance matrices to better capture complex data structures.

Findings

Models outperform previous methods on simulated data

Effective in bioinformatics and text analytics applications

Demonstrates improved data fit with flexible covariance structures

Abstract

Biclustering is used for simultaneous clustering of the observations and variables when there is no group structure known \textit{a priori}. It is being increasingly used in bioinformatics, text analytics, etc. Previously, biclustering has been introduced in a model-based clustering framework by utilizing a structure similar to a mixture of factor analyzers. In such models, observed variables $\mathbf{X}$ are modelled using a latent variable $\mathbf{U}$ that is assumed to be from $N(\mathbf{0}, \mathbf{I})$. Clustering of variables is introduced by imposing constraints on the entries of the factor loading matrix to be 0 and 1 that results in a block diagonal covariance matrices. However, this approach is overly restrictive as off-diagonal elements in the blocks of the covariance matrices can only be 1 which can lead to unsatisfactory model fit on complex data. Here, the latent variable $\mathbf{U}$ is assumed to be from a $N(\mathbf{0}, \mathbf{T})$ where $\mathbf{T}$ is a diagonal matrix. This ensures that the off-diagonal terms in the block matrices within the covariance matrices are non-zero and not restricted to be 1. This leads to a superior model fit on complex data. A family of models are developed by imposing constraints on the components of the covariance matrix. For parameter estimation, an alternating expectation conditional maximization (AECM) algorithm is used. Finally, the proposed method is illustrated using simulated and real datasets.