ILSIC: Corpora for Identifying Indian Legal Statutes from Queries by Laypeople

TL;DR

This paper introduces ILSIC, a new corpus of Indian legal queries from laypeople, enabling better understanding and modeling of informal legal questions distinct from court judgments.

Contribution

The creation of ILSIC, a comprehensive dataset of laypeople legal queries and court judgments, and extensive benchmarking to analyze differences and transfer learning potential.

Findings

Models trained on court judgments perform poorly on laypeople queries.

Transfer learning can improve performance in certain scenarios.

Fine-grained analysis reveals variations across query categories and statute frequencies.

Abstract

Legal Statute Identification (LSI) for a given situation is one of the most fundamental tasks in Legal NLP. This task has traditionally been modeled using facts from court judgments as input queries, due to their abundance. However, in practical settings, the input queries are likely to be informal and asked by laypersons, or non-professionals. While a few laypeople LSI datasets exist, there has been little research to explore the differences between court and laypeople data for LSI. In this work, we create ILSIC, a corpus of laypeople queries covering 500+ statutes from Indian law. Additionally, the corpus also contains court case judgements to enable researchers to effectively compare between court and laypeople data for LSI. We conducted extensive experiments on our corpus, including benchmarking over the laypeople dataset using zero and few-shot inference, retrieval-augmented generation and supervised fine-tuning. We observe that models trained purely on court judgements are ineffective during test on laypeople queries, while transfer learning from court to laypeople data can be beneficial in certain scenarios. We also conducted fine-grained analyses of our results in terms of categories of queries and frequency of statutes.