The BIAS Detection Framework: Bias Detection in Word Embeddings and Language Models for European Languages

TL;DR

The paper introduces the BIAS Detection Framework, a tool designed to identify societal biases in European language models and word embeddings, addressing linguistic and geographic specificities.

Contribution

It presents novel bias detection methods tailored for European languages, with a comprehensive framework architecture and publicly available code.

Findings

Framework effectively detects societal biases in European language models

Addresses linguistic and geographic particularities in bias detection

Code is available for ongoing research and development

Abstract

The project BIAS: Mitigating Diversity Biases of AI in the Labor Market is a four-year project funded by the European commission and supported by the Swiss State Secretariat for Education, Research and Innovation (SERI). As part of the project, novel bias detection methods to identify societal bias in language models and word embeddings in European languages are developed, with particular attention to linguistic and geographic particularities. This technical report describes the overall architecture and components of the BIAS Detection Framework. The code described in this technical report is available and will be updated and expanded continuously with upcoming results from the BIAS project. The details about the datasets for the different languages are described in corresponding papers at scientific venues.