Can Human Sex Be Learned Using Only 2D Body Keypoint Estimations?

TL;DR

This paper presents a simple, real-time system for human sex recognition using only 2D joint keypoints from images, achieving 77% accuracy on a public dataset with robustness to noisy detections.

Contribution

It introduces a fully automated, lightweight deep learning approach for sex classification based solely on 2D body keypoints, demonstrating effectiveness and real-time capability.

Findings

Achieved 77% accuracy on PETA dataset.

Analyzed impact of noisy keypoints on classification performance.

Provided insights into factors affecting accuracy.

Abstract

In this paper, we analyze human male and female sex recognition problem and present a fully automated classification system using only 2D keypoints. The keypoints represent human joints. A keypoint set consists of 15 joints and the keypoint estimations are obtained using an OpenPose 2D keypoint detector. We learn a deep learning model to distinguish males and females using the keypoints as input and binary labels as output. We use two public datasets in the experimental section - 3DPeople and PETA. On PETA dataset, we report a 77% accuracy. We provide model performance details on both PETA and 3DPeople. To measure the effect of noisy 2D keypoint detections on the performance, we run separate experiments on 3DPeople ground truth and noisy keypoint data. Finally, we extract a set of factors that affect the classification accuracy and propose future work. The advantage of the approach is that the input is small and the architecture is simple, which enables us to run many experiments and keep the real-time performance in inference. The source code, with the experiments and data preparation scripts, are available on GitHub (https://github.com/kristijanbartol/human-sex-classifier).