AppellateGen: A Benchmark for Appellate Legal Judgment Generation

TL;DR

AppellateGen introduces a benchmark dataset and a multi-agent system for second-instance legal judgment generation, emphasizing reasoning over initial verdicts and evidentiary updates, highlighting challenges for current LLMs.

Contribution

The paper presents a new appellate judgment dataset and a multi-agent system to improve legal reasoning modeling in second-instance trials.

Findings

SLMAS enhances logical consistency in judgments.

Current LLMs struggle with complex appellate reasoning.

The dataset and code are publicly available.

Abstract

Legal judgment generation is a critical task in legal intelligence. However, existing research in legal judgment generation has predominantly focused on first-instance trials, relying on static fact-to-verdict mappings while neglecting the dialectical nature of appellate (second-instance) review. To address this, we introduce AppellateGen, a benchmark for second-instance legal judgment generation comprising 7,351 case pairs. The task requires models to draft legally binding judgments by reasoning over the initial verdict and evidentiary updates, thereby modeling the causal dependency between trial stages. We further propose a judicial Standard Operating Procedure (SOP)-based Legal Multi-Agent System (SLMAS) to simulate judicial workflows, which decomposes the generation process into discrete stages of issue identification, retrieval, and drafting. Experimental results indicate that while SLMAS improves logical consistency, the complexity of appellate reasoning remains a substantial challenge for current LLMs. The dataset and code are publicly available at: https://anonymous.4open.science/r/AppellateGen-5763.