ClaimGen-CN: A Large-scale Chinese Dataset for Legal Claim Generation

TL;DR

This paper introduces ClaimGen-CN, a large-scale Chinese dataset for legal claim generation, evaluates current models' performance, and highlights the need for specialized development to improve factual accuracy and clarity.

Contribution

The paper creates the first Chinese legal claim generation dataset, designs a tailored evaluation metric, and provides a comprehensive zero-shot assessment of existing language models.

Findings

Current models struggle with factual accuracy.

Models lack expressive clarity in generated claims.

The dataset will be publicly available for future research.

Abstract

Legal claims refer to the plaintiff's demands in a case and are essential to guiding judicial reasoning and case resolution. While many works have focused on improving the efficiency of legal professionals, the research on helping non-professionals (e.g., plaintiffs) remains unexplored. This paper explores the problem of legal claim generation based on the given case's facts. First, we construct ClaimGen-CN, the first dataset for Chinese legal claim generation task, from various real-world legal disputes. Additionally, we design an evaluation metric tailored for assessing the generated claims, which encompasses two essential dimensions: factuality and clarity. Building on this, we conduct a comprehensive zero-shot evaluation of state-of-the-art general and legal-domain large language models. Our findings highlight the limitations of the current models in factual precision and expressive clarity, pointing to the need for more targeted development in this domain. To encourage further exploration of this important task, we will make the dataset publicly available.