MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding

TL;DR

MAUD is a new expert-annotated dataset for legal NLP focused on merger agreements, enabling improved reading comprehension models and serving as a benchmark for legal and NLP research.

Contribution

It introduces the first expert-annotated merger agreement dataset, facilitating legal NLP research and model development with over 39,000 examples and annotations.

Findings

Transformer models outperform random baselines

Models perform well on most questions but need improvement on complex cases

MAUD serves as a valuable benchmark for legal NLP

Abstract

Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address this challenge, we introduce the Merger Agreement Understanding Dataset (MAUD), an expert-annotated reading comprehension dataset based on the American Bar Association's 2021 Public Target Deal Points Study, with over 39,000 examples and over 47,000 total annotations. Our fine-tuned Transformer baselines show promising results, with models performing well above random on most questions. However, on a large subset of questions, there is still room for significant improvement. As the only expert-annotated merger agreement dataset, MAUD is valuable as a benchmark for both the legal profession and the NLP community.