Multilingual Coarse Political Stance Classification of Media. The Editorial Line of a ChatGPT and Bard Newspaper

TL;DR

This study develops a multilingual classifier to identify political bias in news outlets and AI-generated articles, revealing that AI stance evolves over time and varies across languages, similar to traditional media.

Contribution

It introduces a multilingual dataset with coarse stance annotations and demonstrates that classifiers can effectively identify editorial lines in both traditional and AI-generated news articles.

Findings

Classifiers accurately identify political bias across four languages.

AI-generated articles show evolving editorial lines over time.

AI stance varies among languages, mirroring traditional media patterns.

Abstract

Neutrality is difficult to achieve and, in politics, subjective. Traditional media typically adopt an editorial line that can be used by their potential readers as an indicator of the media bias. Several platforms currently rate news outlets according to their political bias. The editorial line and the ratings help readers in gathering a balanced view of news. But in the advent of instruction-following language models, tasks such as writing a newspaper article can be delegated to computers. Without imposing a biased persona, where would an AI-based news outlet lie within the bias ratings? In this work, we use the ratings of authentic news outlets to create a multilingual corpus of news with coarse stance annotations (Left and Right) along with automatically extracted topic annotations. We show that classifiers trained on this data are able to identify the editorial line of most unseen newspapers in English, German, Spanish and Catalan. We then apply the classifiers to 101 newspaper-like articles written by ChatGPT and Bard in the 4 languages at different time periods. We observe that, similarly to traditional newspapers, ChatGPT editorial line evolves with time and, being a data-driven system, the stance of the generated articles differs among languages.