Reframing Tax Law Entailment as Analogical Reasoning

TL;DR

This paper redefines statutory reasoning as an analogy task, expanding dataset size and interpretability, and demonstrates that combining retrieval with analogy models can improve legal reasoning AI performance.

Contribution

It introduces a novel analogy-based framing for statutory reasoning, significantly enlarges the dataset, and combines retrieval with analogy models for better legal AI reasoning.

Findings

Analogy task is as challenging as original statutory reasoning for NLP models.

The new dataset increases size by two orders of magnitude.

Combining retrieval with analogy models shows progress in legal reasoning accuracy.

Abstract

Statutory reasoning refers to the application of legislative provisions to a series of case facts described in natural language. We re-frame statutory reasoning as an analogy task, where each instance of the analogy task involves a combination of two instances of statutory reasoning. This increases the dataset size by two orders of magnitude, and introduces an element of interpretability. We show that this task is roughly as difficult to Natural Language Processing models as the original task. Finally, we come back to statutory reasoning, solving it with a combination of a retrieval mechanism and analogy models, and showing some progress on prior comparable work.