A Cross-Cultural Assessment of Human Ability to Detect LLM-Generated Fake News about South Africa

TL;DR

This study explores how cultural proximity influences the ability to detect AI-generated fake news, revealing that cultural familiarity improves true news verification but may hinder fake news detection.

Contribution

It provides a cross-cultural comparison of fake news detection capabilities, highlighting the impact of cultural context on verification strategies and biases.

Findings

South Africans better at detecting true news about their country

Participants from other countries more accurate at identifying fake news

Cultural familiarity influences reliance on content knowledge versus linguistic features

Abstract

This study investigates how cultural proximity affects the ability to detect AI-generated fake news by comparing South African participants with those from other nationalities. As large language models increasingly enable the creation of sophisticated fake news, understanding human detection capabilities becomes crucial, particularly across different cultural contexts. We conducted a survey where 89 participants (56 South Africans, 33 from other nationalities) evaluated 10 true South African news articles and 10 AI-generated fake versions. Results reveal an asymmetric pattern: South Africans demonstrated superior performance in detecting true news about their country (40% deviation from ideal rating) compared to other participants (52%), but performed worse at identifying fake news (62% vs. 55%). This difference may reflect South Africans' higher overall trust in news sources. Our analysis further shows that South Africans relied more on content knowledge and contextual understanding when judging credibility, while participants from other countries emphasised formal linguistic features such as grammar and structure. Overall, the deviation from ideal rating was similar between groups (51% vs. 53%), suggesting that cultural familiarity appears to aid verification of authentic information but may also introduce bias when evaluating fabricated content. These insights contribute to understanding cross-cultural dimensions of misinformation detection and inform strategies for combating AI-generated fake news in increasingly globalised information ecosystems where content crosses cultural and geographical boundaries.