Chinese Court Simulation with LLM-Based Agent System

TL;DR

This paper introduces SimCourt, a comprehensive LLM-based Chinese court simulation framework that faithfully models trial procedures and roles, improving legal judgment prediction and surpassing real trial annotations in accuracy.

Contribution

The paper presents the first systematic Chinese court simulation framework using LLMs, covering all trial stages and roles, with enhanced agent capabilities for legal analysis.

Findings

SimCourt improves legal judgment prediction accuracy.

Agents' responses outperform real judges and lawyers in many scenarios.

Framework faithfully replicates Chinese court procedures.

Abstract

Mock trial has long served as an important platform for legal professional training and education. It not only helps students learn about realistic trial procedures, but also provides practical value for case analysis and judgment prediction. Traditional mock trials are difficult to access by the public because they rely on professional tutors and human participants. Fortunately, the rise of large language models (LLMs) provides new opportunities for creating more accessible and scalable court simulations. While promising, existing research mainly focuses on agent construction while ignoring the systematic design and evaluation of court simulations, which are actually more important for the credibility and usage of court simulation in practice. To this end, we present the first court simulation framework -- SimCourt -- based on the real-world procedure structure of Chinese courts. Our framework replicates all 5 core stages of a Chinese trial and incorporates 5 courtroom roles, faithfully following the procedural definitions in China. To simulate trial participants with different roles, we propose and craft legal agents equipped with memory, planning, and reflection abilities. Experiment on legal judgment prediction show that our framework can generate simulated trials that better guide the system to predict the imprisonment, probation, and fine of each case. Further annotations by human experts show that agents' responses under our simulation framework even outperformed judges and lawyers from the real trials in many scenarios. These further demonstrate the potential of LLM-based court simulation.