YinYang-Net: Complementing Face and Body Information for Wild Gender Recognition

TL;DR

YinYang-Net is a novel model that dynamically combines face and body cues to improve gender recognition in wild surveillance conditions, outperforming existing methods and addressing challenges of facial detection failures.

Contribution

The paper introduces YinYang-Net, a new model that effectively fuses facial and body information for gender recognition in wild scenarios, along with new datasets derived from existing PAR datasets.

Findings

YinYang-Net achieves state-of-the-art gender recognition accuracy.

The model reduces prediction errors by up to 24% on frontal samples.

New datasets enable better evaluation of soft biometrics in wild conditions.

Abstract

Soft biometrics inference in surveillance scenarios is a topic of interest for various applications, particularly in security-related areas. However, soft biometric analysis is not extensively reported in wild conditions. In particular, previous works on gender recognition report their results in face datasets, with relatively good image quality and frontal poses. Given the uncertainty of the availability of the facial region in wild conditions, we consider that these methods are not adequate for surveillance settings. To overcome these limitations, we: 1) present frontal and wild face versions of three well-known surveillance datasets; and 2) propose YinYang-Net (YY-Net), a model that effectively and dynamically complements facial and body information, which makes it suitable for gender recognition in wild conditions. The frontal and wild face datasets derive from widely used Pedestrian Attribute Recognition (PAR) sets (PETA, PA-100K, and RAP), using a pose-based approach to filter the frontal samples and facial regions. This approach retrieves the facial region of images with varying image/subject conditions, where the state-of-the-art face detectors often fail. YY-Net combines facial and body information through a learnable fusion matrix and a channel-attention sub-network, focusing on the most influential body parts according to the specific image/subject features. We compare it with five PAR methods, consistently obtaining state-of-the-art results on gender recognition, and reducing the prediction errors by up to 24% in frontal samples. The announced PAR datasets versions and YY-Net serve as the basis for wild soft biometrics classification and are available in https://github.com/Tiago-Roxo.