Deep Learning with Nonparametric Clustering

TL;DR

This paper introduces a novel deep learning framework that integrates nonparametric clustering with deep belief networks, enabling effective unsupervised clustering with automatic model complexity inference.

Contribution

It presents a unified model combining deep feature learning and nonparametric clustering, trained efficiently in an online manner, which was less explored before.

Findings

Outperforms competitive clustering baselines.

Effectively learns features for clustering.

Automatically infers model complexity.

Abstract

Clustering is an essential problem in machine learning and data mining. One vital factor that impacts clustering performance is how to learn or design the data representation (or features). Fortunately, recent advances in deep learning can learn unsupervised features effectively, and have yielded state of the art performance in many classification problems, such as character recognition, object recognition and document categorization. However, little attention has been paid to the potential of deep learning for unsupervised clustering problems. In this paper, we propose a deep belief network with nonparametric clustering. As an unsupervised method, our model first leverages the advantages of deep learning for feature representation and dimension reduction. Then, it performs nonparametric clustering under a maximum margin framework -- a discriminative clustering model and can be trained online efficiently in the code space. Lastly model parameters are refined in the deep belief network. Thus, this model can learn features for clustering and infer model complexity in an unified framework. The experimental results show the advantage of our approach over competitive baselines.