Gender prediction using limited Twitter Data

TL;DR

This study demonstrates that fine-tuned BERT models can accurately predict Twitter users' gender with as few as 20-200 tweets, enabling quick gender detection with limited data.

Contribution

It shows that pre-trained transformer models like BERT can be effectively fine-tuned for gender prediction on social media with minimal data, which was previously underexplored.

Findings

BERT achieves 80% F1 with 200 tweets per person.

Performance drops to 70% F1 with only 20 tweets.

Even small datasets enable reasonably accurate gender prediction.

Abstract

Transformer models have shown impressive performance on a variety of NLP tasks. Off-the-shelf, pre-trained models can be fine-tuned for specific NLP classification tasks, reducing the need for large amounts of additional training data. However, little research has addressed how much data is required to accurately fine-tune such pre-trained transformer models, and how much data is needed for accurate prediction. This paper explores the usability of BERT (a Transformer model for word embedding) for gender prediction on social media. Forensic applications include detecting gender obfuscation, e.g. males posing as females in chat rooms. A Dutch BERT model is fine-tuned on different samples of a Dutch Twitter dataset labeled for gender, varying in the number of tweets used per person. The results show that finetuning BERT contributes to good gender classification performance (80% F1) when finetuned on only 200 tweets per person. But when using just 20 tweets per person, the performance of our classifier deteriorates non-steeply (to 70% F1). These results show that even with relatively small amounts of data, BERT can be fine-tuned to accurately help predict the gender of Twitter users, and, consequently, that it is possible to determine gender on the basis of just a low volume of tweets. This opens up an operational perspective on the swift detection of gender.