Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

TL;DR

This paper introduces Multi-VAE, a novel variational autoencoder framework that learns disentangled, explainable visual representations for multi-view clustering by separating common and view-specific features, leading to improved clustering accuracy.

Contribution

The paper proposes a new VAE-based model with a discrete view-common variable and continuous view-peculiar variables, enhancing multi-view clustering by disentangling shared and view-specific information.

Findings

Achieves superior clustering performance over state-of-the-art methods.

Produces disentangled and explainable visual representations.

Effectively separates common and view-specific features.

Abstract

Multi-view clustering, a long-standing and important research problem, focuses on mining complementary information from diverse views. However, existing works often fuse multiple views' representations or handle clustering in a common feature space, which may result in their entanglement especially for visual representations. To address this issue, we present a novel VAE-based multi-view clustering framework (Multi-VAE) by learning disentangled visual representations. Concretely, we define a view-common variable and multiple view-peculiar variables in the generative model. The prior of view-common variable obeys approximately discrete Gumbel Softmax distribution, which is introduced to extract the common cluster factor of multiple views. Meanwhile, the prior of view-peculiar variable follows continuous Gaussian distribution, which is used to represent each view's peculiar visual factors. By controlling the mutual information capacity to disentangle the view-common and view-peculiar representations, continuous visual information of multiple views can be separated so that their common discrete cluster information can be effectively mined. Experimental results demonstrate that Multi-VAE enjoys the disentangled and explainable visual representations, while obtaining superior clustering performance compared with state-of-the-art methods.