Simple Deep Random Model Ensemble

TL;DR

This paper introduces a simple deep random model ensemble (DRME) that leverages random model ensembles and clustering perspectives to improve unsupervised representation learning and clustering accuracy.

Contribution

It presents a novel, straightforward deep learning algorithm based on random model ensembles and clustering ensemble concepts, offering an alternative to traditional deep learning methods.

Findings

DRME outperforms five representation learning methods on 19 datasets.

DRME accurately detects the number of natural clusters.

The approach simplifies deep learning by avoiding EM optimization in k-means.

Abstract

Representation learning and unsupervised learning are two central topics of machine learning and signal processing. Deep learning is one of the most effective unsupervised representation learning approach. The main contributions of this paper to the topics are as follows. (i) We propose to view the representative deep learning approaches as special cases of the knowledge reuse framework of clustering ensemble. (ii) We propose to view sparse coding when used as a feature encoder as the consensus function of clustering ensemble, and view dictionary learning as the training process of the base clusterings of clustering ensemble. (ii) Based on the above two views, we propose a very simple deep learning algorithm, named deep random model ensemble (DRME). It is a stack of random model ensembles. Each random model ensemble is a special k-means ensemble that discards the expectation-maximization optimization of each base k-means but only preserves the default initialization method of the base k-means. (iv) We propose to select the most powerful representation among the layers by applying DRME to clustering where the single-linkage is used as the clustering algorithm. Moreover, the DRME based clustering can also detect the number of the natural clusters accurately. Extensive experimental comparisons with 5 representation learning methods on 19 benchmark data sets demonstrate the effectiveness of DRME.