Beyond Algorithmic Bias: A Socio-Computational Interrogation of the Google Search by Image Algorithm

TL;DR

This study critically examines Google's image search algorithm, revealing it reproduces societal biases and stereotypes, especially favoring white males, and discusses how social theories can improve algorithmic accountability.

Contribution

It introduces a socio-computational framework to audit biases in Google's image search, linking algorithmic outputs to societal and cultural structures.

Findings

Algorithm reproduces white male patriarchal stereotypes.

Faces bias towards white males with more positive descriptions.

Algorithmic representations reflect societal and cultural biases.

Abstract

We perform a socio-computational interrogation of the google search by image algorithm, a main component of the google search engine. We audit the algorithm by presenting it with more than 40 thousands faces of all ages and more than four races and collecting and analyzing the assigned labels with the appropriate statistical tools. We find that the algorithm reproduces white male patriarchal structures, often simplifying, stereotyping and discriminating females and non-white individuals, while providing more diverse and positive descriptions of white men. By drawing from Bourdieu's theory of cultural reproduction, we link these results to the attitudes of the algorithm's designers, owners, and the dataset the algorithm was trained on. We further underpin the problematic nature of the algorithm by using the ethnographic practice of studying-up: We show how the algorithm places individuals at the top of the tech industry within the socio-cultural reality that they shaped, many times creating biased representations of them. We claim that the use of social-theoretic frameworks such as the above are able to contribute to improved algorithmic accountability, algorithmic impact assessment and provide additional and more critical depth in algorithmic bias and auditing studies. Based on the analysis, we discuss the scientific and design implications and provide suggestions for alternative ways to design just socioalgorithmic systems.