Leveraging Text Guidance for Enhancing Demographic Fairness in Gender Classification

TL;DR

This paper introduces text-guided methods using image captions to improve fairness and reduce bias in gender classification from facial images, without relying on demographic labels.

Contribution

It proposes novel multimodal strategies, Image Text Matching and Image Text Fusion, to enhance fairness and interpretability in gender classification models.

Findings

Effective bias mitigation across gender and racial groups

Improved accuracy demonstrated on benchmark datasets

Text guidance reduces demographic disparities

Abstract

In the quest for fairness in artificial intelligence, novel approaches to enhance it in facial image based gender classification algorithms using text guided methodologies are presented. The core methodology involves leveraging semantic information from image captions during model training to improve generalization capabilities. Two key strategies are presented: Image Text Matching (ITM) guidance and Image Text fusion. ITM guidance trains the model to discern fine grained alignments between images and texts to obtain enhanced multimodal representations. Image text fusion combines both modalities into comprehensive representations for improved fairness. Exensive experiments conducted on benchmark datasets demonstrate these approaches effectively mitigate bias and improve accuracy across gender racial groups compared to existing methods. Additionally, the unique integration of textual guidance underscores an interpretable and intuitive training paradigm for computer vision systems. By scrutinizing the extent to which semantic information reduces disparities, this research offers valuable insights into cultivating more equitable facial analysis algorithms. The proposed methodologies contribute to addressing the pivotal challenge of demographic bias in gender classification from facial images. Furthermore, this technique operates in the absence of demographic labels and is application agnostic.