From Same Photo: Cheating on Visual Kinship Challenges
Mitchell Dawson, Andrew Zisserman, Christoffer Nell{\aa}ker

TL;DR
This paper reveals that kinship verification models often rely on faces from the same photograph as an unintended signal, which can lead to misleadingly high accuracy, questioning the validity of current benchmarks.
Contribution
The study demonstrates that a simple classifier based on whether faces come from the same photo can achieve near state-of-the-art results on kinship verification datasets, exposing a data artifact.
Findings
Faces from the same photograph are a strong unintended signal.
Naive classifiers can outperform existing models using this artifact.
Current kinship datasets may overestimate true kinship signals.
Abstract
With the propensity for deep learning models to learn unintended signals from data sets there is always the possibility that the network can `cheat' in order to solve a task. In the instance of data sets for visual kinship verification, one such unintended signal could be that the faces are cropped from the same photograph, since faces from the same photograph are more likely to be from the same family. In this paper we investigate the influence of this artefactual data inference in published data sets for kinship verification. To this end, we obtain a large dataset, and train a CNN classifier to determine if two faces are from the same photograph or not. Using this classifier alone as a naive classifier of kinship, we demonstrate near state of the art results on five public benchmark data sets for kinship verification - achieving over 90% accuracy on one of them. Thus, we conclude…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · Evolutionary Psychology and Human Behavior · Generative Adversarial Networks and Image Synthesis
