Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases

TL;DR

This paper explores paragraph-level rationale extraction in legal texts, introducing a new dataset and analyzing the effectiveness of various regularization constraints, highlighting challenges and opportunities for future research.

Contribution

It presents a novel paragraph-level rationale extraction task in legal texts, along with a new dataset and analysis of regularization constraints, including a new singularity constraint.

Findings

Some rationale constraints are not beneficial at paragraph level

Re-formulation of constraints improves multi-label rationale extraction

The new singularity constraint enhances rationale quality

Abstract

Interpretability or explainability is an emerging research field in NLP. From a user-centric point of view, the goal is to build models that provide proper justification for their decisions, similar to those of humans, by requiring the models to satisfy additional constraints. To this end, we introduce a new application on legal text where, contrary to mainstream literature targeting word-level rationales, we conceive rationales as selected paragraphs in multi-paragraph structured court cases. We also release a new dataset comprising European Court of Human Rights cases, including annotations for paragraph-level rationales. We use this dataset to study the effect of already proposed rationale constraints, i.e., sparsity, continuity, and comprehensiveness, formulated as regularizers. Our findings indicate that some of these constraints are not beneficial in paragraph-level rationale extraction, while others need re-formulation to better handle the multi-label nature of the task we consider. We also introduce a new constraint, singularity, which further improves the quality of rationales, even compared with noisy rationale supervision. Experimental results indicate that the newly introduced task is very challenging and there is a large scope for further research.