Visual Psychophysics for Making Face Recognition Algorithms More Explainable

TL;DR

This paper proposes using visual psychophysics, involving controlled stimulus manipulation, to better understand and explain face recognition algorithms, enhancing interpretability of their failure modes.

Contribution

It introduces a comprehensive psychophysical methodology for analyzing face recognition models, applied to both deep learning and traditional approaches.

Findings

Psychophysical procedures reveal specific failure causes.

Method applied successfully to various face recognition models.

Enhanced understanding of model behavior and limitations.

Abstract

Scientific fields that are interested in faces have developed their own sets of concepts and procedures for understanding how a target model system (be it a person or algorithm) perceives a face under varying conditions. In computer vision, this has largely been in the form of dataset evaluation for recognition tasks where summary statistics are used to measure progress. While aggregate performance has continued to improve, understanding individual causes of failure has been difficult, as it is not always clear why a particular face fails to be recognized, or why an impostor is recognized by an algorithm. Importantly, other fields studying vision have addressed this via the use of visual psychophysics: the controlled manipulation of stimuli and careful study of the responses they evoke in a model system. In this paper, we suggest that visual psychophysics is a viable methodology for making face recognition algorithms more explainable. A comprehensive set of procedures is developed for assessing face recognition algorithm behavior, which is then deployed over state-of-the-art convolutional neural networks and more basic, yet still widely used, shallow and handcrafted feature-based approaches.