Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning

TL;DR

This paper demonstrates that using textual entailment models, which learn explicit language logic, can significantly reduce social biases in sentence representations compared to traditional similarity-based models.

Contribution

It introduces the idea that textual entailment models mitigate stereotypes more effectively than standard pretrained sentence encoders.

Findings

Textual entailment models reduce stereotypical biases.

Pretrained similarity-based models internalize social stereotypes.

Explicit logic learning improves community recognition.

Abstract

Due to their similarity-based learning objectives, pretrained sentence encoders often internalize stereotypical assumptions that reflect the social biases that exist within their training corpora. In this paper, we describe several kinds of stereotypes concerning different communities that are present in popular sentence representation models, including pretrained next sentence prediction and contrastive sentence representation models. We compare such models to textual entailment models that learn language logic for a variety of downstream language understanding tasks. By comparing strong pretrained models based on text similarity with textual entailment learning, we conclude that the explicit logic learning with textual entailment can significantly reduce bias and improve the recognition of social communities, without an explicit de-biasing process