Summarizing User-generated Textual Content: Motivation and Methods for Fairness in Algorithmic Summaries

TL;DR

This paper highlights the importance of fairness in user-generated text summarization, revealing biases in existing algorithms and proposing new methods to ensure equitable representation of diverse social groups.

Contribution

It introduces the first framework for fair summarization of user-generated content, addressing social group biases and proposing fairness-preserving algorithms.

Findings

Existing algorithms often under-represent certain social groups.

Fairness issues lead to unequal exposure of groups in summaries.

Proposed algorithms improve fairness without sacrificing summary quality.

Abstract

As the amount of user-generated textual content grows rapidly, text summarization algorithms are increasingly being used to provide users a quick overview of the information content. Traditionally, summarization algorithms have been evaluated only based on how well they match human-written summaries (e.g. as measured by ROUGE scores). In this work, we propose to evaluate summarization algorithms from a completely new perspective that is important when the user-generated data to be summarized comes from different socially salient user groups, e.g. men or women, Caucasians or African-Americans, or different political groups (Republicans or Democrats). In such cases, we check whether the generated summaries fairly represent these different social groups. Specifically, considering that an extractive summarization algorithm selects a subset of the textual units (e.g. microblogs) in the original data for inclusion in the summary, we investigate whether this selection is fair or not. Our experiments over real-world microblog datasets show that existing summarization algorithms often represent the socially salient user-groups very differently compared to their distributions in the original data. More importantly, some groups are frequently under-represented in the generated summaries, and hence get far less exposure than what they would have obtained in the original data. To reduce such adverse impacts, we propose novel fairness-preserving summarization algorithms which produce high-quality summaries while ensuring fairness among various groups. To our knowledge, this is the first attempt to produce fair text summarization, and is likely to open up an interesting research direction.