BillSum: A Corpus for Automatic Summarization of US Legislation

TL;DR

This paper introduces BillSum, a novel dataset for automatic summarization of US legislation, and evaluates extractive summarization methods, demonstrating transferability across different legislative domains.

Contribution

It provides the first dataset for legislative summarization and benchmarks methods, highlighting challenges and transferability of models across state and federal bills.

Findings

Neural and traditional extractive methods perform effectively on legislative data.

Models trained on Congressional bills can be adapted to California bills.

BillSum dataset presents unique challenges due to legislative language.

Abstract

Automatic summarization methods have been studied on a variety of domains, including news and scientific articles. Yet, legislation has not previously been considered for this task, despite US Congress and state governments releasing tens of thousands of bills every year. In this paper, we introduce BillSum, the first dataset for summarization of US Congressional and California state bills (https://github.com/FiscalNote/BillSum). We explain the properties of the dataset that make it more challenging to process than other domains. Then, we benchmark extractive methods that consider neural sentence representations and traditional contextual features. Finally, we demonstrate that models built on Congressional bills can be used to summarize California bills, thus, showing that methods developed on this dataset can transfer to states without human-written summaries.