Probabilistically Aligned View-unaligned Clustering with Adaptive Template Selection

TL;DR

This paper introduces a novel probabilistic approach for clustering unaligned multi-view data by integrating permutation derivation with bipartite graph modeling, enabling effective alignment and clustering.

Contribution

It proposes PAVuC-ATS, a new method that combines adaptive template selection and Markov chain-based permutation derivation for view-unaligned clustering.

Findings

Outperforms baseline methods on six benchmark datasets.

Theoretical and experimental validation of convergence.

Effective alignment of unaligned multi-view data achieved.

Abstract

In most existing multi-view modeling scenarios, cross-view correspondence (CVC) between instances of the same target from different views, like paired image-text data, is a crucial prerequisite for effortlessly deriving a consistent representation. Nevertheless, this premise is frequently compromised in certain applications, where each view is organized and transmitted independently, resulting in the view-unaligned problem (VuP). Restoring CVC of unaligned multi-view data is a challenging and highly demanding task that has received limited attention from the research community. To tackle this practical challenge, we propose to integrate the permutation derivation procedure into the bipartite graph paradigm for view-unaligned clustering, termed Probabilistically Aligned View-unaligned Clustering with Adaptive Template Selection (PAVuC-ATS). Specifically, we learn consistent anchors and view-specific graphs by the bipartite graph, and derive permutations applied to the unaligned graphs by reformulating the alignment between two latent representations as a 2-step transition of a Markov chain with adaptive template selection, thereby achieving the probabilistic alignment. The convergence of the resultant optimization problem is validated both experimentally and theoretically. Extensive experiments on six benchmark datasets demonstrate the superiority of the proposed PAVuC-ATS over the baseline methods.