Discriminative Similarity for Clustering and Semi-Supervised Learning

TL;DR

This paper introduces a discriminative similarity learning framework that improves clustering and semi-supervised learning by optimizing pairwise similarities through kernel classifiers and generalization error minimization.

Contribution

It proposes a novel framework that learns discriminative similarity by minimizing classifier generalization error over hypothetical labelings, enhancing clustering and semi-supervised learning.

Findings

Discriminative similarity is expressed as pairwise data similarity weighted by kernel classifier parameters.

The framework effectively integrates similarity learning with classifier generalization analysis.

New clustering and semi-supervised methods are developed based on the learned discriminative similarity.

Abstract

Similarity-based clustering and semi-supervised learning methods separate the data into clusters or classes according to the pairwise similarity between the data, and the pairwise similarity is crucial for their performance. In this paper, we propose a novel discriminative similarity learning framework which learns discriminative similarity for either data clustering or semi-supervised learning. The proposed framework learns classifier from each hypothetical labeling, and searches for the optimal labeling by minimizing the generalization error of the learned classifiers associated with the hypothetical labeling. Kernel classifier is employed in our framework. By generalization analysis via Rademacher complexity, the generalization error bound for the kernel classifier learned from hypothetical labeling is expressed as the sum of pairwise similarity between the data from different classes, parameterized by the weights of the kernel classifier. Such pairwise similarity serves as the discriminative similarity for the purpose of clustering and semi-supervised learning, and discriminative similarity with similar form can also be induced by the integrated squared error bound for kernel density classification. Based on the discriminative similarity induced by the kernel classifier, we propose new clustering and semi-supervised learning methods.