Learning Sparsity and Block Diagonal Structure in Multi-View Mixture Models

TL;DR

This paper introduces methods for learning the sparsity and block diagonal structure of cluster membership matrices in multi-view mixture models, enabling better understanding of heterogeneous data sources in integrative analysis.

Contribution

It develops a penalized likelihood approach for sparsity pattern estimation and a novel constrained likelihood method using symmetric graph Laplacian for block diagonal structures.

Findings

Effective in simulations and real data applications

Accurately identifies sparse and block diagonal structures

Extends naturally to multiple data views

Abstract

Scientific studies increasingly collect multiple modalities of data to investigate a phenomenon from several perspectives. In integrative data analysis it is important to understand how information is heterogeneously spread across these different data sources. To this end, we consider a parametric clustering model for the subjects in a multi-view data set (i.e. multiple sources of data from the same set of subjects) where each view marginally follows a mixture model. In the case of two views, the dependence between them is captured by a cluster membership matrix parameter and we aim to learn the structure of this matrix (e.g. the zero pattern). First, we develop a penalized likelihood approach to estimate the sparsity pattern of the cluster membership matrix. For the specific case of block diagonal structures, we develop a constrained likelihood formulation where this matrix is constrained to be block diagonal up to permutations of the rows and columns. To enforce block diagonal constraints we propose a novel optimization approach based on the symmetric graph Laplacian. We demonstrate the performance of these methods through both simulations and applications to data sets from cancer genetics and neuroscience. Both methods naturally extend to multiple views.