Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Bias

TL;DR

This paper evaluates biases in LLM-generated summaries of European Parliament debates, revealing systematic underrepresentation of certain speakers and proposing a hierarchical summarisation method to mitigate positional bias.

Contribution

It introduces an attribution-aware evaluation framework and a hierarchical summarisation approach to measure and reduce inclusion biases in parliamentary summaries.

Findings

Speakers in the middle of debates are systematically excluded.

Non-English speakers are less faithfully represented.

Hierarchical summarisation improves positional bias across models.

Abstract

The The use of Large language models (LLMs) to summarise parliamentary proceedings presents a promising means of increasing the accessibility of democratic participation. However, as these systems increasingly mediate access to political information -- filtering and framing content before it reaches users -- there are important fairness considerations to address. In this work, we evaluate 5 LLMs (both proprietary and open-weight) in the summarisation of plenary debates from the European Parliament to investigate the representational biases that emerge in this context. We develop an attribution-aware evaluation framework to measure speaker-level inclusion and mis-representation in debate summaries. Across all models and experiments, we find that speakers are less accurately represented in the final summary on the basis of (i) their speaking-order (speeches in the middle of the debate were systematically excluded), (ii) language spoken (non-English speakers were less faithfully represented), and (iii) political affiliations (better outcomes for left-of-centre parties). We further show how biases in these contexts can be decomposed to distinguish inclusion bias (systematic omission) from hallucination bias (systematic misrepresentation), and explore the effect of different mitigation strategies. Prompting strategies do not affect these biases. Instead, we propose a hierarchical summarisation method that decomposes the task into simpler extraction and aggregation steps, which we show significantly improves the positional/speaking-order bias across all models. These findings underscore the need for domain-sensitive evaluation metrics and ethical oversight in the deployment of LLMs for multilingual democratic applications.