A dependent partition-valued process for multitask clustering and time evolving network modelling

TL;DR

This paper introduces a Gaussian process-based partition-valued process for flexible multitask clustering and dynamic network modeling, allowing partitions to vary smoothly over arbitrary covariates like space and time.

Contribution

It presents a novel non-Markovian partition process applicable to arbitrary covariate spaces, enabling improved multitask clustering and time-evolving network analysis.

Findings

Successfully identified cancer subtypes from cellular data.

Discovered regulatory modules across diverse populations.

Detected time-varying community structures in social networks.

Abstract

The fundamental aim of clustering algorithms is to partition data points. We consider tasks where the discovered partition is allowed to vary with some covariate such as space or time. One approach would be to use fragmentation-coagulation processes, but these, being Markov processes, are restricted to linear or tree structured covariate spaces. We define a partition-valued process on an arbitrary covariate space using Gaussian processes. We use the process to construct a multitask clustering model which partitions datapoints in a similar way across multiple data sources, and a time series model of network data which allows cluster assignments to vary over time. We describe sampling algorithms for inference and apply our method to defining cancer subtypes based on different types of cellular characteristics, finding regulatory modules from gene expression data from multiple human populations, and discovering time varying community structure in a social network.