From Verification to Amplification: Auditing Reverse Image Search as Algorithmic Gatekeeping in Visual Misinformation Fact-checking

TL;DR

This paper audits Google reverse image search to understand its role in visual misinformation verification, revealing significant irrelevant and misleading results, with challenges in visibility for debunking content and a temporal quality fluctuation.

Contribution

It provides a systematic analysis of Google RIS's effectiveness in visual misinformation verification, highlighting its limitations and the dynamics of search result quality over time.

Findings

RIS returns many irrelevant and repeated misinformation results

Debunking content is less than 30% of search results

Search result quality follows an inverted U-shape over time

Abstract

As visual misinformation becomes increasingly prevalent, platform algorithms act as intermediaries that curate information for users' verification practices. Yet, it remains unclear how algorithmic gatekeeping tools, such as reverse image search (RIS), shape users' information exposure during fact-checking. This study systematically audits Google RIS by reversely searching newly identified misleading images over a 15-day window and analyzing 34,486 collected top-ranked search results. We find that Google RIS returns a substantial volume of irrelevant information and repeated misinformation, whereas debunking content constitutes less than 30% of search results. Debunking content faces visibility challenges in rankings amid repeated misinformation and irrelevant information. Our findings also indicate an inverted U-shaped curve of RIS results page quality over time, likely due to search engine "data voids" when visual falsehoods first appear. These findings contribute to scholarship of visual misinformation verification, and extend algorithmic gatekeeping research to the visual domain.