AppealCase: A Dataset and Benchmark for Civil Case Appeal Scenarios
Yuting Huang, Meitong Guo, Yiquan Wu, Ang Li, Xiaozhong Liu, Keting Yin, Changlong Sun, Fei Wu, Kun Kuang

TL;DR
The paper introduces AppealCase, a comprehensive dataset and benchmark for analyzing civil case appeal scenarios, addressing a gap in LegalAI research by focusing on appellate review processes.
Contribution
It provides a large, annotated dataset for civil appeals and proposes five new LegalAI tasks, enabling systematic evaluation of models in appellate case analysis.
Findings
Models achieve less than 50% F1 on judgment reversal prediction
The dataset covers 91 civil case categories with detailed annotations
Current models struggle with the complexity of appeal scenarios
Abstract
Recent advances in LegalAI have primarily focused on individual case judgment analysis, often overlooking the critical appellate process within the judicial system. Appeals serve as a core mechanism for error correction and ensuring fair trials, making them highly significant both in practice and in research. To address this gap, we present the AppealCase dataset, consisting of 10,000 pairs of real-world, matched first-instance and second-instance documents across 91 categories of civil cases. The dataset also includes detailed annotations along five dimensions central to appellate review: judgment reversals, reversal reasons, cited legal provisions, claim-level decisions, and whether there is new information in the second instance. Based on these annotations, we propose five novel LegalAI tasks and conduct a comprehensive evaluation across 20 mainstream models. Experimental results…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsArtificial Intelligence in Law · Law, Economics, and Judicial Systems · Legal Education and Practice Innovations
