Discriminative Similarity for Data Clustering

TL;DR

This paper introduces CDS, a novel clustering method that learns discriminative similarity by minimizing classifier generalization error, improving clustering performance through a theoretically grounded approach.

Contribution

It proposes a new discriminative similarity measure learned via unsupervised kernel classification, with theoretical analysis and a practical clustering algorithm CDSK.

Findings

Discriminative similarity improves clustering accuracy.

Theoretical bounds relate similarity to classification error.

Experimental results validate the effectiveness of CDSK.

Abstract

Similarity-based clustering methods separate data into clusters according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance. In this paper, we propose {\em Clustering by Discriminative Similarity (CDS)}, a novel method which learns discriminative similarity for data clustering. CDS learns an unsupervised similarity-based classifier from each data partition, and searches for the optimal partition of the data by minimizing the generalization error of the learnt classifiers associated with the data partitions. By generalization analysis via Rademacher complexity, the generalization error bound for the unsupervised similarity-based classifier is expressed as the sum of discriminative similarity between the data from different classes. It is proved that the derived discriminative similarity can also be induced by the integrated squared error bound for kernel density classification. In order to evaluate the performance of the proposed discriminative similarity, we propose a new clustering method using a kernel as the similarity function, CDS via unsupervised kernel classification (CDSK), with its effectiveness demonstrated by experimental results.