fMBN-E: Efficient Unsupervised Network Structure Ensemble and Selection for Clustering

TL;DR

This paper introduces fMBN-E, an efficient ensemble and selection method for deep clustering that automatically determines optimal network structures, achieving state-of-the-art results with minimal hyperparameter tuning.

Contribution

It proposes a novel ensemble and selection framework for multilayer bootstrap networks that is faster and maintains high performance without manual hyperparameter tuning.

Findings

fMBN-E achieves state-of-the-art clustering performance.

fMBN-E is hundreds of times faster than the original MBN-E.

The methods outperform existing deep clustering and ensemble techniques.

Abstract

It is known that unsupervised nonlinear dimensionality reduction and clustering is sensitive to the selection of hyperparameters, particularly for deep learning based methods, which hinders its practical use. How to select a proper network structure that may be dramatically different in different applications is a hard issue for deep models, given little prior knowledge of data. In this paper, we aim to automatically determine the optimal network structure of a deep model, named multilayer bootstrap networks (MBN), via simple ensemble learning and selection techniques. Specifically, we first propose an MBN ensemble (MBN-E) algorithm which concatenates the sparse outputs of a set of MBN base models with different network structures into a new representation. Then, we take the new representation produced by MBN-E as a reference for selecting the optimal MBN base models. Moreover, we propose a fast version of MBN-E (fMBN-E), which is not only theoretically even faster than a single standard MBN but also does not increase the estimation error of MBN-E. Importantly, MBN-E and its ensemble selection techniques maintain the simple formulation of MBN that is based on one-nearest-neighbor learning. Empirically, comparing to a number of advanced deep clustering methods and as many as 20 representative unsupervised ensemble learning and selection methods, the proposed methods reach the state-of-the-art performance without manual hyperparameter tuning. fMBN-E is empirically even hundreds of times faster than MBN-E without suffering performance degradation. The applications to image segmentation and graph data mining further demonstrate the advantage of the proposed methods.