Learning the kernel matrix by resampling

TL;DR

This paper introduces a novel kernel learning method using a nonparametric density estimator based on k-centroids clustering, producing a sparse, parameter-insensitive kernel matrix that improves spectral clustering results.

Contribution

The paper presents a new kernel learning approach that generates a sparse, nonlinear kernel matrix from clustering-based density estimation, with advantages of simplicity and reduced parameter sensitivity.

Findings

Kernel outperforms Gaussian RBF in spectral clustering

Method is insensitive to free parameters

Produces sparse, nonlinear kernel matrices

Abstract

In this abstract paper, we introduce a new kernel learning method by a nonparametric density estimator. The estimator consists of a group of k-centroids clusterings. Each clustering randomly selects data points with randomly selected features as its centroids, and learns a one-hot encoder by one-nearest-neighbor optimization. The estimator generates a sparse representation for each data point. Then, we construct a nonlinear kernel matrix from the sparse representation of data. One major advantage of the proposed kernel method is that it is relatively insensitive to its free parameters, and therefore, it can produce reasonable results without parameter tuning. Another advantage is that it is simple. We conjecture that the proposed method can find its applications in many learning tasks or methods where sparse representation or kernel matrix is explored. In this preliminary study, we have applied the kernel matrix to spectral clustering. Our experimental results demonstrate that the kernel generated by the proposed method outperforms the well-tuned Gaussian RBF kernel. This abstract paper is used to protect the idea, full versions will be updated later.