Multi-Agent Simulator Drives Language Models for Legal Intensive Interaction

TL;DR

This paper presents MASER, a multi-agent simulation framework that generates synthetic legal scenario data to enhance LLM training, along with MILE, a benchmark for evaluating LLMs in dynamic legal interactions.

Contribution

Introduces MASER, a scalable multi-agent simulator for legal scenarios, and MILE, a benchmark for assessing LLM performance in interactive legal contexts.

Findings

MASER effectively generates consistent synthetic legal data.

LLMs show improved performance on the MILE benchmark.

Framework enhances legal intelligence capabilities of LLMs.

Abstract

Large Language Models (LLMs) have significantly advanced legal intelligence, but the scarcity of scenario data impedes the progress toward interactive legal scenarios. This paper introduces a Multi-agent Legal Simulation Driver (MASER) to scalably generate synthetic data by simulating interactive legal scenarios. Leveraging real-legal case sources, MASER ensures the consistency of legal attributes between participants and introduces a supervisory mechanism to align participants' characters and behaviors as well as addressing distractions. A Multi-stage Interactive Legal Evaluation (MILE) benchmark is further constructed to evaluate LLMs' performance in dynamic legal scenarios. Extensive experiments confirm the effectiveness of our framework.