From Perceptions To Evidence: Detecting AI-Generated Content In Turkish News Media With A Fine-Tuned Bert Classifier

TL;DR

This study develops and evaluates a Turkish-specific BERT classifier to empirically detect AI-generated content in Turkish news media, revealing that approximately 2.5% of articles are AI-rewritten, with stable detection performance over time.

Contribution

It introduces the first empirical, data-driven method for detecting AI-generated news content in Turkish media using a fine-tuned BERT model.

Findings

Achieved 0.9708 F1 score in classification accuracy.

Detected approximately 2.5% of articles as AI-rewritten.

Model performance remained stable across sources and time periods.

Abstract

The rapid integration of large language models into newsroom workflows has raised urgent questions about the prevalence of AI-generated content in online media. While computational studies have begun to quantify this phenomenon in English-language outlets, no empirical investigation exists for Turkish news media, where existing research remains limited to qualitative interviews with journalists or fake news detection. This study addresses that gap by fine-tuning a Turkish-specific BERT model (dbmdz/bert-base-turkish-cased) on a labeled dataset of 3,600 articles from three major Turkish outlets with distinct editorial orientations for binary classification of AI-rewritten content. The model achieves 0.9708 F1 score on the held-out test set with symmetric precision and recall across both classes. Subsequent deployment on over 3,500 unseen articles spanning between 2023 and 2026 reveals consistent cross-source and temporally stable classification patterns, with mean prediction confidence exceeding 0.96 and an estimated 2.5 percentage of examined news content rewritten or revised by LLMs on average. To the best of our knowledge, this is the first study to move beyond self-reported journalist perceptions toward empirical, data-driven measurement of AI usage in Turkish news media.