Similarity Learning via Kernel Preserving Embedding

TL;DR

This paper introduces a novel similarity learning framework that preserves kernel relations by minimizing kernel matrix reconstruction error, leading to improved clustering and potential applications in low-dimensional embedding.

Contribution

It proposes a new kernel preserving similarity learning method that enhances data relations retention, surpassing existing data reconstruction approaches.

Findings

Significant improvement in clustering performance.

Framework is general and applicable to various similarity tasks.

Enables effective embedding of high-dimensional data into low-dimensional space.

Abstract

Data similarity is a key concept in many data-driven applications. Many algorithms are sensitive to similarity measures. To tackle this fundamental problem, automatically learning of similarity information from data via self-expression has been developed and successfully applied in various models, such as low-rank representation, sparse subspace learning, semi-supervised learning. However, it just tries to reconstruct the original data and some valuable information, e.g., the manifold structure, is largely ignored. In this paper, we argue that it is beneficial to preserve the overall relations when we extract similarity information. Specifically, we propose a novel similarity learning framework by minimizing the reconstruction error of kernel matrices, rather than the reconstruction error of original data adopted by existing work. Taking the clustering task as an example to evaluate our method, we observe considerable improvements compared to other state-of-the-art methods. More importantly, our proposed framework is very general and provides a novel and fundamental building block for many other similarity-based tasks. Besides, our proposed kernel preserving opens up a large number of possibilities to embed high-dimensional data into low-dimensional space.