Comparing diversity, negativity, and stereotypes in Chinese-language AI technologies: an investigation of Baidu, Ernie and Qwen

TL;DR

This study compares biases, negativity, and stereotypes in Chinese-language AI tools, revealing that language models show more diverse views but also more negativity and stereotypes than search engines, emphasizing the need for fairness.

Contribution

First comprehensive analysis of social biases in Chinese AI tools, highlighting differences between search engines and language models and emphasizing global fairness considerations.

Findings

Language models show greater diversity of views.

Baidu and Qwen generate more negative content.

Stereotypes are prevalent and sometimes offensive.

Abstract

Large Language Models (LLMs) and search engines have the potential to perpetuate biases and stereotypes by amplifying existing prejudices in their training data and algorithmic processes, thereby influencing public perception and decision-making. While most work has focused on Western-centric AI technologies, we study Chinese-based tools by investigating social biases embedded in the major Chinese search engine, Baidu, and two leading LLMs, Ernie and Qwen. Leveraging a dataset of 240 social groups across 13 categories describing Chinese society, we collect over 30k views encoded in the aforementioned tools by prompting them for candidate words describing such groups. We find that language models exhibit a larger variety of embedded views compared to the search engine, although Baidu and Qwen generate negative content more often than Ernie. We also find a moderate prevalence of stereotypes embedded in the language models, many of which potentially promote offensive and derogatory views. Our work highlights the importance of promoting fairness and inclusivity in AI technologies with a global perspective.