Efficient k-means with Individual Fairness via Exponential Tilting

TL;DR

This paper introduces tilted k-means (TKM), a novel algorithm that incorporates exponential tilting to achieve individual fairness in clustering, improving fairness and efficiency over existing methods.

Contribution

The paper proposes a new tilted SSE objective and an efficient optimization algorithm for fair clustering, along with a novel fairness metric and theoretical guarantees.

Findings

TKM outperforms state-of-the-art methods in effectiveness.

TKM achieves better fairness by reducing variance in distances.

The algorithm has linear time complexity with dataset size.

Abstract

In location-based resource allocation scenarios, the distances between each individual and the facility are desired to be approximately equal, thereby ensuring fairness. Individually fair clustering is often employed to achieve the principle of treating all points equally, which can be applied in these scenarios. This paper proposes a novel algorithm, tilted k-means (TKM), aiming to achieve individual fairness in clustering. We integrate the exponential tilting into the sum of squared errors (SSE) to formulate a novel objective function called tilted SSE. We demonstrate that the tilted SSE can generalize to SSE and employ the coordinate descent and first-order gradient method for optimization. We propose a novel fairness metric, the variance of the distances within each cluster, which can alleviate the Matthew Effect typically caused by existing fairness metrics. Our theoretical analysis demonstrates that the well-known k-means++ incurs a multiplicative error of O(k log k), and we establish the convergence of TKM under mild conditions. In terms of fairness, we prove that the variance generated by TKM decreases with a scaled hyperparameter. In terms of efficiency, we demonstrate the time complexity is linear with the dataset size. Our experiments demonstrate that TKM outperforms state-of-the-art methods in effectiveness, fairness, and efficiency.