Cluster-Adaptive Feature Extraction and its Theoretical Foundation with Minkowski Weighted k-Means

TL;DR

This paper introduces a theoretical foundation for Minkowski weighted k-means, revealing how feature weights depend on dispersion, and proposes Cluster-Adaptive Feature Extraction (CAFE) to enhance unsupervised feature extraction.

Contribution

The paper provides a theoretical analysis of mwk-means, deriving bounds and structure of feature weights, and introduces CAFE, a novel method leveraging these insights for improved feature extraction.

Findings

Theoretical bounds for mwk-means objective function.

Feature weights depend on relative dispersion following a power-law.

CAFE improves traditional feature extraction methods in noisy, unsupervised settings.

Abstract

The Minkowski weighted $k$-means ($mwk$-means) algorithm extends classical $k$-means by incorporating feature weights and a Minkowski distance. We first show that the $mwk$-means objective can be expressed as a power-mean aggregation of within-cluster dispersions, with the order determined by the Minkowski exponent $p$. This formulation reveals how $p$ controls the transition between selective and uniform use of features. Using this representation, we derive bounds for the objective function and characterise the structure of the feature weights, showing that they depend only on relative dispersion and follow a power-law relationship with dispersion ratios. This leads to explicit guarantees on the suppression of high-dispersion features, and we establish convergence of the algorithm. Building on these theoretical results, we introduce Cluster-Adaptive Feature Extraction (CAFE), a method that uses the $mwk$-means feature weights to rescale the data prior to unsupervised feature extraction. We prove that this rescaling reverses the within-cluster dispersion ordering, suppressing noisy features and amplifying informative ones. Numerous experiments conducted under controlled within-cluster noise show that CAFE consistently improves the results of traditional feature extraction methods.