Multi-document Summarization through Multi-document Event Relation Graph Reasoning in LLMs: a case study in Framing Bias Mitigation

TL;DR

This paper presents a novel multi-document event relation graph reasoning approach in LLMs to generate neutralized summaries, effectively mitigating media bias by leveraging event relations and opinions across multiple articles.

Contribution

It introduces a multi-document event relation graph and two strategies to incorporate it into LLMs for bias mitigation in summarization, a novel approach in this domain.

Findings

Effective reduction of lexical bias in summaries.

Improved content preservation alongside bias mitigation.

Both automatic and human evaluations confirm success.

Abstract

Media outlets are becoming more partisan and polarized nowadays. Most previous work focused on detecting media bias. In this paper, we aim to mitigate media bias by generating a neutralized summary given multiple articles presenting different ideological views. Motivated by the critical role of events and event relations in media bias detection, we propose to increase awareness of bias in LLMs via multi-document events reasoning and use a multi-document event relation graph to guide the summarization process. This graph contains rich event information useful to reveal bias: four common types of in-doc event relations to reflect content framing bias, cross-doc event coreference relation to reveal content selection bias, and event-level moral opinions to highlight opinionated framing bias. We further develop two strategies to incorporate the multi-document event relation graph for neutralized summarization. Firstly, we convert a graph into natural language descriptions and feed the textualized graph into LLMs as a part of a hard text prompt. Secondly, we encode the graph with graph attention network and insert the graph embedding into LLMs as a soft prompt. Both automatic evaluation and human evaluation confirm that our approach effectively mitigates both lexical and informational media bias, and meanwhile improves content preservation.