Fairness in Visual Clustering: A Novel Transformer Clustering Approach

TL;DR

This paper introduces a new transformer-based clustering method that promotes fairness by reducing demographic bias, using a novel loss function and attention mechanism to improve cluster purity and fairness without relying on balanced annotated data.

Contribution

It proposes a novel loss function and a cross-attention mechanism to enhance fairness and cluster purity in deep unsupervised clustering models.

Findings

Improves clustering accuracy on large-scale datasets.

Reduces demographic bias in clustering results.

Enhances fairness across multiple sensitive attributes.

Abstract

Promoting fairness for deep clustering models in unsupervised clustering settings to reduce demographic bias is a challenging goal. This is because of the limitation of large-scale balanced data with well-annotated labels for sensitive or protected attributes. In this paper, we first evaluate demographic bias in deep clustering models from the perspective of cluster purity, which is measured by the ratio of positive samples within a cluster to their correlation degree. This measurement is adopted as an indication of demographic bias. Then, a novel loss function is introduced to encourage a purity consistency for all clusters to maintain the fairness aspect of the learned clustering model. Moreover, we present a novel attention mechanism, Cross-attention, to measure correlations between multiple clusters, strengthening faraway positive samples and improving the purity of clusters during the learning process. Experimental results on a large-scale dataset with numerous attribute settings have demonstrated the effectiveness of the proposed approach on both clustering accuracy and fairness enhancement on several sensitive attributes.