
TL;DR
This study investigates whether large language models exhibit genre-based biases in news credibility assessment, finding that some models are more prone to misclassify entertainment news as fake, with implications for evaluation practices.
Contribution
It reveals genre-specific false-positive biases in LLMs and demonstrates that prompt framing can mitigate these biases in certain models, highlighting the importance of genre-aware evaluation.
Findings
DeepSeek-V3.2 and GPT-5.2 show significant genre bias in false positives.
Style-swap experiments indicate bias is not solely due to stylistic differences.
Prompt framing reduces false positives for some models without affecting recall.
Abstract
Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed standards across journalistic genres. We examine whether zero-shot LLMs are more likely to misclassify legitimate entertainment news as fake than legitimate hard news, using a within-dataset design on GossipCop from FakeNewsNet. Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both ), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference. A style-swap experiment yields only limited and inconsistent changes, suggesting that the asymmetry is not reducible to stylistic register alone. Prompt-based mitigation is likewise possible but not generic: framing the model as an…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
