WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report

TL;DR

This paper introduces WisdomInterrogatory (LuWen), an open-source Chinese legal language model that leverages continual pre-training, supervised fine-tuning, and retrieval-augmented generation to excel in various legal tasks.

Contribution

The paper presents a novel legal LLM, LuWen, built on Baichuan with specialized techniques and comprehensive evaluation across multiple legal NLP tasks.

Findings

LuWen outperforms strong baselines in legal judgment prediction.

Retrieval-augmented generation improves legal question answering accuracy.

Continual pre-training on legal data enhances model domain adaptation.

Abstract

Large language models have demonstrated remarkable capabilities across a wide range of natural language processing tasks, yet their application in the legal domain remains challenging due to the specialized terminology, complex reasoning requirements, and rapidly evolving legal knowledge involved. In this paper, we present WisdomInterrogatory (LuWen), an open-source Chinese legal language model built upon the Baichuan foundation model through three key techniques: continual pre-training on a large-scale legal corpus, supervised fine-tuning with carefully curated legal instruction data, and retrieval-augmented generation integrated with a comprehensive legal knowledge base. We evaluate LuWen on five representative legal tasks spanning both prediction and generation settings, including legal judgment prediction, judicial examination, legal text summarization, law article question answering, and judicial decision reasoning. Experimental results show that LuWen outperforms several strong baselines, demonstrating the effectiveness of our approach in adapting general-purpose language models to the legal domain.