Deep Embedded K-Means Clustering

TL;DR

This paper introduces DEKM, a deep clustering method that transforms autoencoder embeddings to better reveal cluster structures, optimizing representations and clustering iteratively without relying on reconstruction loss.

Contribution

The paper proposes a novel transformation of autoencoder embeddings using eigenvectors to enhance cluster structure detection and a greedy optimization method for representation learning.

Findings

DEKM achieves state-of-the-art clustering performance on real-world datasets.

Transforming embedding space improves the visibility of cluster structures.

Alternating optimization enhances both representation quality and clustering accuracy.

Abstract

Recently, deep clustering methods have gained momentum because of the high representational power of deep neural networks (DNNs) such as autoencoder. The key idea is that representation learning and clustering can reinforce each other: Good representations lead to good clustering while good clustering provides good supervisory signals to representation learning. Critical questions include: 1) How to optimize representation learning and clustering? 2) Should the reconstruction loss of autoencoder be considered always? In this paper, we propose DEKM (for Deep Embedded K-Means) to answer these two questions. Since the embedding space generated by autoencoder may have no obvious cluster structures, we propose to further transform the embedding space to a new space that reveals the cluster-structure information. This is achieved by an orthonormal transformation matrix, which contains the eigenvectors of the within-class scatter matrix of K-means. The eigenvalues indicate the importance of the eigenvectors' contributions to the cluster-structure information in the new space. Our goal is to increase the cluster-structure information. To this end, we discard the decoder and propose a greedy method to optimize the representation. Representation learning and clustering are alternately optimized by DEKM. Experimental results on the real-world datasets demonstrate that DEKM achieves state-of-the-art performance.