Fair Abstractive Summarization of Diverse Perspectives
Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo, Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen, McKeown, Rui Zhang

TL;DR
This paper addresses the challenge of fair abstractive summarization by defining fairness as balanced representation of diverse perspectives, proposing new metrics, evaluating multiple LLMs, and suggesting methods to improve fairness.
Contribution
It introduces formal fairness metrics for abstractive summarization, evaluates state-of-the-art LLMs on diverse datasets, and proposes techniques to enhance fairness in summaries.
Findings
Both models and human references show low fairness in summaries.
Fairness is influenced by source perspective distribution and model biases.
Proposed methods improve fairness without significantly degrading summary quality.
Abstract
People from different social and demographic groups express diverse perspectives and conflicting opinions on a broad set of topics such as product reviews, healthcare, law, and politics. A fair summary should provide a comprehensive coverage of diverse perspectives without underrepresenting certain groups. However, current work in summarization metrics and Large Language Models (LLMs) evaluation has not explored fair abstractive summarization. In this paper, we systematically investigate fair abstractive summarization for user-generated data. We first formally define fairness in abstractive summarization as not underrepresenting perspectives of any groups of people, and we propose four reference-free automatic metrics by measuring the differences between target and source perspectives. We evaluate nine LLMs, including three GPT models, four LLaMA models, PaLM 2, and Claude, on six…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsComputational and Text Analysis Methods
MethodsSparse Evolutionary Training · Pathways Language Model · LLaMA · Multi-Head Attention · Attention Is All You Need · Linear Layer · Weight Decay · Layer Normalization · Residual Connection · Byte Pair Encoding
