Facial Kinship Verification from remote photoplethysmography

TL;DR

This paper introduces a novel approach to facial kinship verification using remote photoplethysmography signals extracted from facial videos, leveraging a 1D CNN with attention and contrastive loss to improve kinship detection accuracy.

Contribution

It is the first to explore facial kinship verification with vital bio-signals, specifically rPPG, and proposes a new neural network architecture for this task.

Findings

rPPG signals can effectively indicate kinship relations.

The proposed model achieves promising results on the UvANEMO Smile Database.

Using multiple facial ROIs improves kinship verification accuracy.

Abstract

Facial Kinship Verification (FKV) aims at automatically determining whether two subjects have a kinship relation based on human faces. It has potential applications in finding missing children and social media analysis. Traditional FKV faces challenges as it is vulnerable to spoof attacks and raises privacy issues. In this paper, we explore for the first time the FKV with vital bio-signals, focusing on remote Photoplethysmography (rPPG). rPPG signals are extracted from facial videos, resulting in a one-dimensional signal that measures the changes in visible light reflection emitted to and detected from the skin caused by the heartbeat. Specifically, in this paper, we employed a straightforward one-dimensional Convolutional Neural Network (1DCNN) with a 1DCNN-Attention module and kinship contrastive loss to learn the kin similarity from rPPGs. The network takes multiple rPPG signals extracted from various facial Regions of Interest (ROIs) as inputs. Additionally, the 1DCNN attention module is designed to learn and capture the discriminative kin features from feature embeddings. Finally, we demonstrate the feasibility of rPPG to detect kinship with the experiment evaluation on the UvANEMO Smile Database from different kin relations.