Human-Aided Saliency Maps Improve Generalization of Deep Learning

TL;DR

This paper introduces a novel method of incorporating human judgment of salient image regions into training data, significantly improving deep learning model accuracy and generalization in biometric attack detection, especially with limited data.

Contribution

It pioneers encoding human saliency judgments into training data to enhance deep learning performance and generalization in biometric security tasks.

Findings

Models trained with human-encoded saliency outperform traditional methods.

Error rate reduced from 29.78% to 16.37% with the new approach.

Enhanced generalization in leave-one-attack-type-out scenarios.

Abstract

Deep learning has driven remarkable accuracy increases in many computer vision problems. One ongoing challenge is how to achieve the greatest accuracy in cases where training data is limited. A second ongoing challenge is that trained models oftentimes do not generalize well even to new data that is subjectively similar to the training set. We address these challenges in a novel way, with the first-ever (to our knowledge) exploration of encoding human judgement about salient regions of images into the training data. We compare the accuracy and generalization of a state-of-the-art deep learning algorithm for a difficult problem in biometric presentation attack detection when trained on (a) original images with typical data augmentations, and (b) the same original images transformed to encode human judgement about salient image regions. The latter approach results in models that achieve higher accuracy and better generalization, decreasing the error of the LivDet-Iris 2020 winner from 29.78% to 16.37%, and achieving impressive generalization in a leave-one-attack-type-out evaluation scenario. This work opens a new area of study for how to embed human intelligence into training strategies for deep learning to achieve high accuracy and generalization in cases of limited training data.