Chatlaw: A Multi-Agent Collaborative Legal Assistant with Knowledge Graph Enhanced Mixture-of-Experts Large Language Model

TL;DR

Chatlaw is a multi-agent legal assistant that combines knowledge graphs and a mixture-of-experts model to improve accuracy and reduce hallucinations in AI legal services, outperforming GPT-4 in key legal benchmarks.

Contribution

This work introduces a novel multi-agent system with a knowledge graph-enhanced MoE model and SOPs to improve reliability and accuracy in AI legal assistance.

Findings

MoE model outperforms GPT-4 by 7.73% in accuracy on Lawbench

Achieves 11-point improvement in legal professional exam scores

Reduces hallucinations and errors in legal responses

Abstract

AI legal assistants based on Large Language Models (LLMs) can provide accessible legal consulting services, but the hallucination problem poses potential legal risks. This paper presents Chatlaw, an innovative legal assistant utilizing a Mixture-of-Experts (MoE) model and a multi-agent system to enhance the reliability and accuracy of AI-driven legal services. By integrating knowledge graphs with artificial screening, we construct a high-quality legal dataset to train the MoE model. This model utilizes different experts to address various legal issues, optimizing the accuracy of legal responses. Additionally, Standardized Operating Procedures (SOP), modeled after real law firm workflows, significantly reduce errors and hallucinations in legal services. Our MoE model outperforms GPT-4 in the Lawbench and Unified Qualification Exam for Legal Professionals by 7.73% in accuracy and 11 points, respectively, and also surpasses other models in multiple dimensions during real-case consultations, demonstrating our robust capability for legal consultation.