PublicHearingBR: A Brazilian Portuguese Dataset of Public Hearing Transcripts for Summarization of Long Documents

TL;DR

PublicHearingBR is a new Brazilian Portuguese dataset of public hearing transcripts paired with summaries and annotations, facilitating research on long document summarization and natural language inference in Portuguese.

Contribution

The paper introduces a novel dataset, a hybrid summarization baseline system, and discusses evaluation metrics addressing hallucination in large language models for Portuguese.

Findings

Dataset enables evaluation of long document summarization in Portuguese

Hybrid summarization system provides a baseline for future research

Annotated data supports natural language inference tasks

Abstract

This paper introduces PublicHearingBR, a Brazilian Portuguese dataset designed for summarizing long documents. The dataset consists of transcripts of public hearings held by the Brazilian Chamber of Deputies, paired with news articles and structured summaries containing the individuals participating in the hearing and their statements or opinions. The dataset supports the development and evaluation of long document summarization systems in Portuguese. Our contributions include the dataset, a hybrid summarization system to establish a baseline for future studies, and a discussion of evaluation metrics for summarization involving large language models, addressing the challenge of hallucination in the generated summaries. As a result of this discussion, the dataset also includes annotated data to evaluate natural language inference tasks in Portuguese.