Improving Model's Focus Improves Performance of Deep Learning-Based Synthetic Face Detectors

TL;DR

This paper explores how increasing a deep learning model's focus by lowering salience entropy enhances synthetic face detection performance, especially on unseen data, by combining classification, focus, and human-guided saliency in training.

Contribution

It introduces entropy-based loss components to control model focus, demonstrating improved detection of synthetic faces in open-set scenarios.

Findings

Lower salience entropy correlates with better generalization.

Optimal performance achieved by blending classification, focus, and human-guided saliency.

Models with reduced focus entropy outperform those without such control.

Abstract

Deep learning-based models generalize better to unknown data samples after being guided "where to look" by incorporating human perception into training strategies. We made an observation that the entropy of the model's salience trained in that way is lower when compared to salience entropy computed for models training without human perceptual intelligence. Thus the question: does further increase of model's focus, by lowering the entropy of model's class activation map, help in further increasing the performance? In this paper we propose and evaluate several entropy-based new loss function components controlling the model's focus, covering the full range of the level of such control, from none to its "aggressive" minimization. We show, using a problem of synthetic face detection, that improving the model's focus, through lowering entropy, leads to models that perform better in an open-set scenario, in which the test samples are synthesized by unknown generative models. We also show that optimal performance is obtained when the model's loss function blends three aspects: regular classification, low-entropy of the model's focus, and human-guided saliency.