On Convergence of Epanechnikov Mean Shift

TL;DR

This paper provides a theoretical analysis of Epanechnikov Mean Shift, proposes fixes for convergence issues, and demonstrates its effectiveness for clustering large, high-dimensional data sets with better performance than traditional algorithms.

Contribution

It offers the first convergence guarantees for Epanechnikov Mean Shift and introduces practical modifications for large-scale clustering tasks.

Findings

Guaranteed convergence to local maxima of the density estimate.

Effective clustering performance on large, high-dimensional data.

Outperforms Lloyd's K-means and EM algorithms in experiments.

Abstract

Epanechnikov Mean Shift is a simple yet empirically very effective algorithm for clustering. It localizes the centroids of data clusters via estimating modes of the probability distribution that generates the data points, using the `optimal' Epanechnikov kernel density estimator. However, since the procedure involves non-smooth kernel density functions, the convergence behavior of Epanechnikov mean shift lacks theoretical support as of this writing---most of the existing analyses are based on smooth functions and thus cannot be applied to Epanechnikov Mean Shift. In this work, we first show that the original Epanechnikov Mean Shift may indeed terminate at a non-critical point, due to the non-smoothness nature. Based on our analysis, we propose a simple remedy to fix it. The modified Epanechnikov Mean Shift is guaranteed to terminate at a local maximum of the estimated density, which corresponds to a cluster centroid, within a finite number of iterations. We also propose a way to avoid running the Mean Shift iterates from every data point, while maintaining good clustering accuracies under non-overlapping spherical Gaussian mixture models. This further pushes Epanechnikov Mean Shift to handle very large and high-dimensional data sets. Experiments show surprisingly good performance compared to the Lloyd's K-means algorithm and the EM algorithm.