Automated Clustering of High-dimensional Data with a Feature Weighted Mean Shift Algorithm

TL;DR

This paper introduces a feature-weighted mean shift algorithm designed for high-dimensional data, improving clustering performance by learning feature importance while maintaining simplicity and providing theoretical convergence guarantees.

Contribution

A novel feature-weighted mean shift algorithm that extends mean shift clustering to high-dimensional data with proven convergence properties.

Findings

Outperforms traditional mean shift in high-dimensional settings

Maintains computational simplicity of the original algorithm

Demonstrates effectiveness on synthetic and real-world datasets

Abstract

Mean shift is a simple interactive procedure that gradually shifts data points towards the mode which denotes the highest density of data points in the region. Mean shift algorithms have been effectively used for data denoising, mode seeking, and finding the number of clusters in a dataset in an automated fashion. However, the merits of mean shift quickly fade away as the data dimensions increase and only a handful of features contain useful information about the cluster structure of the data. We propose a simple yet elegant feature-weighted variant of mean shift to efficiently learn the feature importance and thus, extending the merits of mean shift to high-dimensional data. The resulting algorithm not only outperforms the conventional mean shift clustering procedure but also preserves its computational simplicity. In addition, the proposed method comes with rigorous theoretical convergence guarantees and a convergence rate of at least a cubic order. The efficacy of our proposal is thoroughly assessed through experimental comparison against baseline and state-of-the-art clustering methods on synthetic as well as real-world datasets.