Learning to Cluster Faces via Transformer

TL;DR

This paper introduces a Face Transformer model that improves face clustering accuracy by leveraging local context and relation encoding, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a novel Face Transformer architecture that decomposes face clustering into relation encoding and linkage prediction, enhancing robustness and accuracy.

Findings

Achieves 91.12% pairwise F-score on MS-Celeb-1M

Outperforms existing methods on face clustering benchmarks

Demonstrates robustness to pose, occlusion, and image quality variations

Abstract

Face clustering is a useful tool for applications like automatic face annotation and retrieval. The main challenge is that it is difficult to cluster images from the same identity with different face poses, occlusions, and image quality. Traditional clustering methods usually ignore the relationship between individual images and their neighbors which may contain useful context information. In this paper, we repurpose the well-known Transformer and introduce a Face Transformer for supervised face clustering. In Face Transformer, we decompose the face clustering into two steps: relation encoding and linkage predicting. Specifically, given a face image, a \textbf{relation encoder} module aggregates local context information from its neighbors and a \textbf{linkage predictor} module judges whether a pair of images belong to the same cluster or not. In the local linkage graph view, Face Transformer can generate more robust node and edge representations compared to existing methods. Experiments on both MS-Celeb-1M and DeepFashion show that our method achieves state-of-the-art performance, e.g., 91.12\% in pairwise F-score on MS-Celeb-1M.