PILOT: Legal Case Outcome Prediction with Case Law

TL;DR

This paper introduces PILOT, a framework for predicting legal case outcomes in case law systems, emphasizing relevant precedent retrieval and temporal context, with improved accuracy demonstrated on a curated dataset.

Contribution

PILOT is a novel framework that addresses the unique challenges of case law outcome prediction by incorporating precedent relevance and temporal dynamics.

Findings

Significant improvement over prior civil law-focused models

Effective retrieval of relevant precedent cases

Handling of legal principle evolution over time

Abstract

Machine learning shows promise in predicting the outcome of legal cases, but most research has concentrated on civil law cases rather than case law systems. We identified two unique challenges in making legal case outcome predictions with case law. First, it is crucial to identify relevant precedent cases that serve as fundamental evidence for judges during decision-making. Second, it is necessary to consider the evolution of legal principles over time, as early cases may adhere to different legal contexts. In this paper, we proposed a new framework named PILOT (PredictIng Legal case OuTcome) for case outcome prediction. It comprises two modules for relevant case retrieval and temporal pattern handling, respectively. To benchmark the performance of existing legal case outcome prediction models, we curated a dataset from a large-scale case law database. We demonstrate the importance of accurately identifying precedent cases and mitigating the temporal shift when making predictions for case law, as our method shows a significant improvement over the prior methods that focus on civil law case outcome predictions.