Video Face Matching using Subset Selection and Clustering of Probabilistic Multi-Region Histograms

TL;DR

This paper investigates the trade-offs in video face recognition between using all detected faces versus selecting subsets, demonstrating that clustering all faces generally yields better accuracy, especially in batch processing scenarios.

Contribution

It introduces a system analyzing performance trade-offs in face subset selection and clustering, providing empirical evidence on MOBIO dataset for improved recognition accuracy.

Findings

Clustering all faces outperforms subset selection in recognition accuracy.

Face detection confidence metric generally yields better results than random or sequential sampling.

Optimal number of faces varies with selection method and dataset subset.

Abstract

Balancing computational efficiency with recognition accuracy is one of the major challenges in real-world video-based face recognition. A significant design decision for any such system is whether to process and use all possible faces detected over the video frames, or whether to select only a few "best" faces. This paper presents a video face recognition system based on probabilistic Multi-Region Histograms to characterise performance trade-offs in: (i) selecting a subset of faces compared to using all faces, and (ii) combining information from all faces via clustering. Three face selection metrics are evaluated for choosing a subset: face detection confidence, random subset, and sequential selection. Experiments on the recently introduced MOBIO dataset indicate that the usage of all faces through clustering always outperformed selecting only a subset of faces. The experiments also show that the face selection metric based on face detection confidence generally provides better recognition performance than random or sequential sampling. Moreover, the optimal number of faces varies drastically across selection metric and subsets of MOBIO. Given the trade-offs between computational effort, recognition accuracy and robustness, it is recommended that face feature clustering would be most advantageous in batch processing (particularly for video-based watchlists), whereas face selection methods should be limited to applications with significant computational restrictions.