Experimenting with Selected Automated Approaches for Bias Analysis

TL;DR

This paper evaluates automated bias analysis models for search engine results, finding current models insufficiently accurate to replace manual bias detection methods.

Contribution

It explores the application of state-of-the-art automated approaches for bias analysis in search results and reports on their limitations.

Findings

Automated models achieved low class-wise F1-scores.

Current approaches are inadequate for reliable bias detection.

Manual analysis remains necessary for bias identification.

Abstract

This work first presents our attempts to establish an automated model using state-of-the-art approaches for analysing bias in search results of Bing and Google. Experimental results indicate that the current class-wise F1-scores of our best model are not sufficient to establish an automated model for bias analysis. Thus, we decided not to continue with this approach.