Legal Rule Induction: Towards Generalizable Principle Discovery from Analogous Judicial Precedents

TL;DR

This paper formalizes the task of Legal Rule Induction using large language models to extract generalizable legal principles from judicial precedents, introducing a new benchmark dataset and analyzing model performance.

Contribution

It defines the novel task of Legal Rule Induction, creates the first benchmark dataset, and evaluates LLMs' ability to extract legal rules from case sets.

Findings

LLMs struggle with over-generalization and hallucination.

Training on the dataset improves LLMs' rule extraction capabilities.

Benchmark dataset enables systematic evaluation of legal rule induction.

Abstract

Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms, social morality, and policy. While computational legal research has advanced in applying established rules to cases, inducing legal rules from judicial decisions remains understudied, constrained by limitations in model inference efficacy and symbolic reasoning capability. The advent of Large Language Models (LLMs) offers unprecedented opportunities for automating the extraction of such latent principles, yet progress is stymied by the absence of formal task definitions, benchmark datasets, and methodologies. To address this gap, we formalize Legal Rule Induction (LRI) as the task of deriving concise, generalizable doctrinal rules from sets of analogous precedents, distilling their shared preconditions, normative behaviors, and legal consequences. We introduce the first LRI benchmark, comprising 5,121 case sets (38,088 Chinese cases in total) for model tuning and 216 expert-annotated gold test sets. Experimental results reveal that: 1) State-of-the-art LLMs struggle with over-generalization and hallucination; 2) Training on our dataset markedly enhances LLMs capabilities in capturing nuanced rule patterns across similar cases.