Enhancing Noise Resilience in Face Clustering via Sparse Differential Transformer

TL;DR

This paper introduces a Sparse Differential Transformer to improve noise resilience in face clustering by enhancing similarity measurement accuracy and reducing irrelevant node influence, leading to state-of-the-art results.

Contribution

We propose a novel Sparse Differential Transformer that effectively eliminates noise and improves the reliability of similarity measurements in face clustering.

Findings

Achieves state-of-the-art performance on MS-Celeb-1M dataset.

Outperforms existing methods in clustering accuracy.

Enhances robustness against noise in face embedding relationships.

Abstract

The method used to measure relationships between face embeddings plays a crucial role in determining the performance of face clustering. Existing methods employ the Jaccard similarity coefficient instead of the cosine distance to enhance the measurement accuracy. However, these methods introduce too many irrelevant nodes, producing Jaccard coefficients with limited discriminative power and adversely affecting clustering performance. To address this issue, we propose a prediction-driven Top-K Jaccard similarity coefficient that enhances the purity of neighboring nodes, thereby improving the reliability of similarity measurements. Nevertheless, accurately predicting the optimal number of neighbors (Top-K) remains challenging, leading to suboptimal clustering results. To overcome this limitation, we develop a Transformer-based prediction model that examines the relationships between the central node and its neighboring nodes near the Top-K to further enhance the reliability of similarity estimation. However, vanilla Transformer, when applied to predict relationships between nodes, often introduces noise due to their overemphasis on irrelevant feature relationships. To address these challenges, we propose a Sparse Differential Transformer (SDT), instead of the vanilla Transformer, to eliminate noise and enhance the model's anti-noise capabilities. Extensive experiments on multiple datasets, such as MS-Celeb-1M, demonstrate that our approach achieves state-of-the-art (SOTA) performance, outperforming existing methods and providing a more robust solution for face clustering.