Pore-scale Image Patch Dataset and A Comparative Evaluation of Pore-scale Facial Features

TL;DR

This paper introduces the PorePatch dataset for pore-scale facial image patches and evaluates deep learning descriptors, revealing their strengths in matching but limitations in 3D reconstruction of weak-texture facial regions.

Contribution

The paper presents a new high-quality pore-scale image patch dataset and a benchmark for evaluating facial descriptors, along with a framework for dataset refinement.

Findings

Deep learning descriptors outperform traditional ones in matching accuracy.

Performance gap between deep learning and traditional descriptors in 3D reconstruction.

The dataset enables more effective evaluation of facial feature descriptors.

Abstract

The weak-texture nature of facial skin regions presents significant challenges for local descriptor matching in applications such as facial motion analysis and 3D face reconstruction. Although deep learning-based descriptors have demonstrated superior performance to traditional hand-crafted descriptors in many applications, the scarcity of pore-scale image patch datasets has hindered their further development in the facial domain. In this paper, we propose the PorePatch dataset, a high-quality pore-scale image patch dataset, and establish a rational evaluation benchmark. We introduce a Data-Model Co-Evolution (DMCE) framework to generate a progressively refined, high-quality dataset from high-resolution facial images. We then train existing SOTA models on our dataset and conduct extensive experiments. Our results show that the SOTA model achieves a FPR95 value of 1.91% on the matching task, outperforming PSIFT (22.41%) by a margin of 20.5%. However, its advantage is diminished in the 3D reconstruction task, where its overall performance is not significantly better than that of traditional descriptors. This indicates that deep learning descriptors still have limitations in addressing the challenges of facial weak-texture regions, and much work remains to be done in this field.