LeCaRDv2: A Large-Scale Chinese Legal Case Retrieval Dataset

TL;DR

LeCaRDv2 is a large-scale, expert-annotated Chinese legal case retrieval dataset with extensive coverage and enriched relevance criteria, designed to improve legal retrieval technologies.

Contribution

It introduces LeCaRDv2, the largest Chinese legal case retrieval dataset with comprehensive relevance criteria and a novel candidate pooling strategy.

Findings

State-of-the-art models perform poorly, indicating room for improvement.

The dataset covers diverse criminal charges and legal relevance aspects.

Expert annotations ensure high reliability and accuracy.

Abstract

As an important component of intelligent legal systems, legal case retrieval plays a critical role in ensuring judicial justice and fairness. However, the development of legal case retrieval technologies in the Chinese legal system is restricted by three problems in existing datasets: limited data size, narrow definitions of legal relevance, and naive candidate pooling strategies used in data sampling. To alleviate these issues, we introduce LeCaRDv2, a large-scale Legal Case Retrieval Dataset (version 2). It consists of 800 queries and 55,192 candidates extracted from 4.3 million criminal case documents. To the best of our knowledge, LeCaRDv2 is one of the largest Chinese legal case retrieval datasets, providing extensive coverage of criminal charges. Additionally, we enrich the existing relevance criteria by considering three key aspects: characterization, penalty, procedure. This comprehensive criteria enriches the dataset and may provides a more holistic perspective. Furthermore, we propose a two-level candidate set pooling strategy that effectively identify potential candidates for each query case. It's important to note that all cases in the dataset have been annotated by multiple legal experts specializing in criminal law. Their expertise ensures the accuracy and reliability of the annotations. We evaluate several state-of-the-art retrieval models at LeCaRDv2, demonstrating that there is still significant room for improvement in legal case retrieval. The details of LeCaRDv2 can be found at the anonymous website https://github.com/anonymous1113243/LeCaRDv2.