Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology

TL;DR

This paper introduces an unsupervised approach to detect bias in contextualized embeddings by leveraging social network information, with a focus on ideological bias and its evolution over time.

Contribution

It presents a novel unsupervised method combining graph neural networks and regularization to identify ideological bias in embeddings without labeled data.

Findings

The ideological subspace encodes evaluative semantics.

It reflects political spectrum shifts during Trump's presidency.

The method effectively uncovers latent bias in social media discussions.

Abstract

We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.