Explainable Model-Agnostic Similarity and Confidence in Face Verification

TL;DR

This paper introduces a novel explainability framework for face verification systems, providing confidence scores and visualization maps to enhance understanding of model decisions, with practical tools and datasets made publicly accessible.

Contribution

It presents a new confidence scoring method and visualization approach for face verification, improving interpretability of deep learning models in this domain.

Findings

Confidence scores correlate with facial feature distances.

Visualization highlights similar and dissimilar facial regions.

Web platform enables user-friendly exploration of explanations.

Abstract

Recently, face recognition systems have demonstrated remarkable performances and thus gained a vital role in our daily life. They already surpass human face verification accountability in many scenarios. However, they lack explanations for their predictions. Compared to human operators, typical face recognition network system generate only binary decisions without further explanation and insights into those decisions. This work focuses on explanations for face recognition systems, vital for developers and operators. First, we introduce a confidence score for those systems based on facial feature distances between two input images and the distribution of distances across a dataset. Secondly, we establish a novel visualization approach to obtain more meaningful predictions from a face recognition system, which maps the distance deviation based on a systematic occlusion of images. The result is blended with the original images and highlights similar and dissimilar facial regions. Lastly, we calculate confidence scores and explanation maps for several state-of-the-art face verification datasets and release the results on a web platform. We optimize the platform for a user-friendly interaction and hope to further improve the understanding of machine learning decisions. The source code is available on GitHub, and the web platform is publicly available at http://explainable-face-verification.ey.r.appspot.com.