Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization

TL;DR

This paper introduces DEPICT, a deep clustering model combining convolutional autoencoders and relative entropy minimization, achieving efficient, scalable, and accurate image clustering without labeled data.

Contribution

The paper presents a joint end-to-end deep clustering framework that integrates autoencoder-based embedding with a novel relative entropy objective, improving efficiency and scalability.

Findings

DEPICT outperforms existing clustering methods in accuracy.

The model achieves faster training times on large-scale datasets.

It effectively prevents overfitting through reconstruction-based regularization.

Abstract

Image clustering is one of the most important computer vision applications, which has been extensively studied in literature. However, current clustering methods mostly suffer from lack of efficiency and scalability when dealing with large-scale and high-dimensional data. In this paper, we propose a new clustering model, called DEeP Embedded RegularIzed ClusTering (DEPICT), which efficiently maps data into a discriminative embedding subspace and precisely predicts cluster assignments. DEPICT generally consists of a multinomial logistic regression function stacked on top of a multi-layer convolutional autoencoder. We define a clustering objective function using relative entropy (KL divergence) minimization, regularized by a prior for the frequency of cluster assignments. An alternating strategy is then derived to optimize the objective by updating parameters and estimating cluster assignments. Furthermore, we employ the reconstruction loss functions in our autoencoder, as a data-dependent regularization term, to prevent the deep embedding function from overfitting. In order to benefit from end-to-end optimization and eliminate the necessity for layer-wise pretraining, we introduce a joint learning framework to minimize the unified clustering and reconstruction loss functions together and train all network layers simultaneously. Experimental results indicate the superiority and faster running time of DEPICT in real-world clustering tasks, where no labeled data is available for hyper-parameter tuning.