TriCon-Fair: Triplet Contrastive Learning for Mitigating Social Bias in Pre-trained Language Models

TL;DR

TriCon-Fair is a contrastive learning framework that effectively reduces social bias in pre-trained language models by decoupling biased and unbiased sample relationships, improving fairness without sacrificing language understanding.

Contribution

It introduces a novel triplet contrastive learning method that decouples positive-negative relationships to better mitigate social bias in language models.

Findings

Reduces social bias more effectively than existing methods

Maintains strong downstream NLP performance

Provides a practical approach for ethical NLP applications

Abstract

The increasing utilization of large language models raises significant concerns about the propagation of social biases, which may result in harmful and unfair outcomes. However, existing debiasing methods treat the biased and unbiased samples independently, thus ignoring their mutual relationship. This oversight enables a hidden negative-positive coupling, where improvements for one group inadvertently compromise the other, allowing residual social bias to persist. In this paper, we introduce TriCon-Fair, a contrastive learning framework that employs a decoupled loss that combines triplet and language modeling terms to eliminate positive-negative coupling. Our TriCon-Fair assigns each anchor an explicitly biased negative and an unbiased positive, decoupling the push-pull dynamics and avoiding positive-negative coupling, and jointly optimizes a language modeling (LM) objective to preserve general capability. Experimental results demonstrate that TriCon-Fair reduces discriminatory output beyond existing debiasing baselines while maintaining strong downstream performance. This suggests that our proposed TriCon-Fair offers a practical and ethical solution for sensitive NLP applications.