Navigating Nuance: In Quest for Political Truth

TL;DR

This paper evaluates the Llama-3 language model's ability to detect political bias using a novel prompting technique on the MBIB benchmark, highlighting challenges and potential improvements in bias detection models.

Contribution

Introduces a novel prompting method for Llama-3 to identify political bias, achieving performance comparable to state-of-the-art models like ConvBERT.

Findings

Llama-3 performs well with the new prompting technique

Transfer learning can enhance bias detection models

Framework achieves state-of-the-art results on MBIB

Abstract

This study investigates the several nuanced rationales for countering the rise of political bias. We evaluate the performance of the Llama-3 (70B) language model on the Media Bias Identification Benchmark (MBIB), based on a novel prompting technique that incorporates subtle reasons for identifying political leaning. Our findings underscore the challenges of detecting political bias and highlight the potential of transfer learning methods to enhance future models. Through our framework, we achieve a comparable performance with the supervised and fully fine-tuned ConvBERT model, which is the state-of-the-art model, performing best among other baseline models for the political bias task on MBIB. By demonstrating the effectiveness of our approach, we contribute to the development of more robust tools for mitigating the spread of misinformation and polarization. Our codes and dataset are made publicly available in github.