IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning
Abhinav Joshi, Shounak Paul, Akshat Sharma, Pawan Goyal and, Saptarshi Ghosh, Ashutosh Modi

TL;DR
IL-TUR is a comprehensive benchmark designed to evaluate NLP models on Indian legal texts across multiple languages, facilitating research in legal understanding and reasoning.
Contribution
This paper introduces IL-TUR, the first benchmark for Indian legal text understanding, including diverse tasks, baseline models, and a public leaderboard for model comparison.
Findings
Baseline models reveal significant gaps in legal understanding.
Multilingual tasks highlight challenges in Indian legal NLP.
IL-TUR enables standardized evaluation of legal NLP systems.
Abstract
Legal systems worldwide are inundated with exponential growth in cases and documents. There is an imminent need to develop NLP and ML techniques for automatically processing and understanding legal documents to streamline the legal system. However, evaluating and comparing various NLP models designed specifically for the legal domain is challenging. This paper addresses this challenge by proposing IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning. IL-TUR contains monolingual (English, Hindi) and multi-lingual (9 Indian languages) domain-specific tasks that address different aspects of the legal system from the point of view of understanding and reasoning over Indian legal documents. We present baseline models (including LLM-based) for each task, outlining the gap between models and the ground truth. To foster further research in the legal domain, we create a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsArtificial Intelligence in Law · Natural Language Processing Techniques · Translation Studies and Practices
