Inferring Gender from Names on the Web: A Comparative Evaluation of Gender Detection Methods

TL;DR

This paper evaluates various automated gender detection methods from names on the web, revealing biases and proposing a novel image-based approach to improve accuracy and reduce bias in demographic inference.

Contribution

It systematically compares existing name-based gender detection methods and introduces a new web image retrieval technique to enhance gender inference accuracy.

Findings

Name-based methods are biased by country of origin.

Combining name-based and image-based methods reduces bias.

The proposed image-based approach improves gender detection accuracy.

Abstract

Computational social scientists often harness the Web as a "societal observatory" where data about human social behavior is collected. This data enables novel investigations of psychological, anthropological and sociological research questions. However, in the absence of demographic information, such as gender, many relevant research questions cannot be addressed. To tackle this problem, researchers often rely on automated methods to infer gender from name information provided on the web. However, little is known about the accuracy of existing gender-detection methods and how biased they are against certain sub-populations. In this paper, we address this question by systematically comparing several gender detection methods on a random sample of scientists for whom we know their full name, their gender and the country of their workplace. We further suggest a novel method that employs web-based image retrieval and gender recognition in facial images in order to augment name-based approaches. Our findings show that the performance of name-based gender detection approaches can be biased towards countries of origin and such biases can be reduced by combining name-based an image-based gender detection methods.