Twitter-Based Gender Recognition Using Transformers

TL;DR

This paper presents a transformer-based approach to predict user gender on Twitter by combining image and text analysis, achieving high accuracy and demonstrating the complementary nature of multimodal data.

Contribution

The study introduces a novel multimodal transformer model that combines Vision Transformers and BERT to improve gender recognition accuracy on social media data.

Findings

Combined model improves accuracy by 6.98% for images and 4.43% for tweets.

Achieved 85.52% accuracy on PAN-2018 dataset.

Multimodal approach outperforms single modality models.

Abstract

Social media contains useful information about people and the society that could help advance research in many different areas (e.g. by applying opinion mining, emotion/sentiment analysis, and statistical analysis) such as business and finance, health, socio-economic inequality and gender vulnerability. User demographics provide rich information that could help study the subject further. However, user demographics such as gender are considered private and are not freely available. In this study, we propose a model based on transformers to predict the user's gender from their images and tweets. We fine-tune a model based on Vision Transformers (ViT) to stratify female and male images. Next, we fine-tune another model based on Bidirectional Encoders Representations from Transformers (BERT) to recognize the user's gender by their tweets. This is highly beneficial, because not all users provide an image that indicates their gender. The gender of such users could be detected form their tweets. The combination model improves the accuracy of image and text classification models by 6.98% and 4.43%, respectively. This shows that the image and text classification models are capable of complementing each other by providing additional information to one another. We apply our method to the PAN-2018 dataset, and obtain an accuracy of 85.52%.