ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation

TL;DR

This paper introduces ILDC, a large annotated corpus of Indian Supreme Court cases, and proposes a task for predicting and explaining court judgments using baseline models and a hierarchical occlusion explainability approach.

Contribution

It provides the first large-scale Indian legal corpus with expert-annotated explanations and formulates the novel task of Court Judgment Prediction and Explanation (CJPE).

Findings

Best prediction accuracy of 78% compared to 94% by legal experts.

Hierarchical occlusion model offers explainability but differs from expert reasoning.

Highlights the complexity and scope for future research in legal judgment prediction.

Abstract

An automated system that could assist a judge in predicting the outcome of a case would help expedite the judicial process. For such a system to be practically useful, predictions by the system should be explainable. To promote research in developing such a system, we introduce ILDC (Indian Legal Documents Corpus). ILDC is a large corpus of 35k Indian Supreme Court cases annotated with original court decisions. A portion of the corpus (a separate test set) is annotated with gold standard explanations by legal experts. Based on ILDC, we propose the task of Court Judgment Prediction and Explanation (CJPE). The task requires an automated system to predict an explainable outcome of a case. We experiment with a battery of baseline models for case predictions and propose a hierarchical occlusion based model for explainability. Our best prediction model has an accuracy of 78% versus 94% for human legal experts, pointing towards the complexity of the prediction task. The analysis of explanations by the proposed algorithm reveals a significant difference in the point of view of the algorithm and legal experts for explaining the judgments, pointing towards scope for future research.