IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators

TL;DR

IndiVec is a versatile media bias detection framework using large language models and vector databases, offering improved adaptability, explainability, and superior performance across diverse datasets.

Contribution

We introduce IndiVec, a general bias detection framework leveraging large language models and fine-grained bias indicators for better adaptability and interpretability.

Findings

Outperforms baseline methods on four political bias datasets.

Demonstrates consistent performance across diverse media sources.

Provides explicit indicators for bias explanation.

Abstract

This study focuses on media bias detection, crucial in today's era of influential social media platforms shaping individual attitudes and opinions. In contrast to prior work that primarily relies on training specific models tailored to particular datasets, resulting in limited adaptability and subpar performance on out-of-domain data, we introduce a general bias detection framework, IndiVec, built upon large language models. IndiVec begins by constructing a fine-grained media bias database, leveraging the robust instruction-following capabilities of large language models and vector database techniques. When confronted with new input for bias detection, our framework automatically selects the most relevant indicator from the vector database and employs majority voting to determine the input's bias label. IndiVec excels compared to previous methods due to its adaptability (demonstrating consistent performance across diverse datasets from various sources) and explainability (providing explicit top-k indicators to interpret bias predictions). Experimental results on four political bias datasets highlight IndiVec's significant superiority over baselines. Furthermore, additional experiments and analysis provide profound insights into the framework's effectiveness.