Off-the-grid: Fast and Effective Hyperparameter Search for Kernel Clustering

TL;DR

This paper introduces a fast, effective method for hyperparameter tuning in kernel k-means clustering, addressing the limitations of grid search with a new algorithm and efficient implementation that improves clustering performance.

Contribution

It derives a theoretical lower bound for kernel parameters, proposes a novel hyperparameter search algorithm, and provides an efficient implementation with guarantees.

Findings

The method effectively identifies useful hyperparameters.

The approach outperforms grid search in efficiency and quality.

Experimental results validate the method's practical utility.

Abstract

Kernel functions are a powerful tool to enhance the $k$-means clustering algorithm via the kernel trick. It is known that the parameters of the chosen kernel function can have a dramatic impact on the result. In supervised settings, these can be tuned via cross-validation, but for clustering this is not straightforward and heuristics are usually employed. In this paper we study the impact of kernel parameters on kernel $k$-means. In particular, we derive a lower bound, tight up to constant factors, below which the parameter of the RBF kernel will render kernel $k$-means meaningless. We argue that grid search can be ineffective for hyperparameter search in this context and propose an alternative algorithm for this purpose. In addition, we offer an efficient implementation based on fast approximate exponentiation with provable quality guarantees. Our experimental results demonstrate the ability of our method to efficiently reveal a rich and useful set of hyperparameter values.