Unsupervised Feature Learning Architecture with Multi-clustering Integration RBM

TL;DR

This paper introduces a novel unsupervised feature learning architecture combining multi-clustering integration with a new RBM variant, enhancing feature representation and generalization for clustering tasks.

Contribution

The paper proposes a multi-clustering integration module and a new MIRBM model that incorporates local clustering guidance into RBM training, improving unsupervised feature learning.

Findings

Outperforms state-of-the-art GraphRBM in clustering accuracy.

Enhances feature representation and generalization capabilities.

Effective integration of multi-clustering results into RBM training.

Abstract

In this paper, we present a novel unsupervised feature learning architecture, which consists of a multi-clustering integration module and a variant of RBM termed multi-clustering integration RBM (MIRBM). In the multi-clustering integration module, we apply three unsupervised K-means, affinity propagation and spectral clustering algorithms to obtain three different clustering partitions (CPs) without any background knowledge or label. Then, an unanimous voting strategy is used to generate a local clustering partition (LCP). The novel MIRBM model is a core feature encoding part of the proposed unsupervised feature learning architecture. The novelty of it is that the LCP as an unsupervised guidance is integrated into one step contrastive divergence (CD1) learning to guide the distribution of the hidden layer features. For the instance in the same LCP cluster, the hidden and reconstructed hidden layer features of the MIRBM model in the proposed architecture tend to constrict together in the training process. Meanwhile, each LCP center tends to disperse from each other as much as possible in the hidden and reconstructed hidden layer during training. The experiments demonstrate that the proposed unsupervised feature learning architecture has more powerful feature representation and generalization capability than the state-of-the-art graph regularized RBM (GraphRBM) for clustering tasks in the Microsoft Research Asia Multimedia (MSRA-MM)2.0 dataset.