LOGAN: Local Group Bias Detection by Clustering

TL;DR

LOGAN is a clustering-based method that detects local biases in machine learning models, providing deeper insights into how biases manifest in specific data regions beyond aggregate performance metrics.

Contribution

It introduces a novel approach for local bias detection in models, addressing limitations of existing corpus-level bias evaluation methods.

Findings

LOGAN effectively identifies local biases in toxicity and object classification tasks.

It enables more detailed analysis of biases in specific data regions.

LOGAN improves understanding of how biases vary across different data instances.

Abstract

Machine learning techniques have been widely used in natural language processing (NLP). However, as revealed by many recent studies, machine learning models often inherit and amplify the societal biases in data. Various metrics have been proposed to quantify biases in model predictions. In particular, several of them evaluate disparity in model performance between protected groups and advantaged groups in the test corpus. However, we argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. In fact, a model with similar aggregated performance between different groups on the entire data may behave differently on instances in a local region. To analyze and detect such local bias, we propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region and allows us to better analyze the biases in model predictions.