Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning

TL;DR

This paper presents D3LM, a diagnostic legal LLM that uses adaptive questions and reinforcement learning to improve legal case analysis and user interaction, supported by a new US case law dataset.

Contribution

Introduction of D3LM, a legal LLM with diagnostic questioning and PURL reinforcement learning, plus a new English legal case dataset for enhanced legal AI applications.

Findings

D3LM outperforms classical LLMs in legal tasks.

Enhanced user experience in legal case analysis.

Effective generation of critical legal questions.

Abstract

The integration of generative Large Language Models (LLMs) into various applications, including the legal domain, has been accelerated by their expansive and versatile nature. However, when facing a legal case, users without a legal background often struggle to formulate professional queries and may inadvertently overlook critical legal factors when presenting their case narrative to LLMs. To address this issue, we propose the Diagnostic Legal Large Language Model (D3LM), which utilizes adaptive lawyer-like diagnostic questions to collect additional case information and then provides high-quality feedback. D3LM incorporates an innovative graph-based Positive-Unlabeled Reinforcement Learning (PURL) algorithm, enabling the generation of critical questions and enhancing user-LLM interactions. Moreover, an integrated LLM-based stopping criterion facilitates precise Court Views Generation (CVG). Our research also introduces a new English-language CVG dataset based on the US case law database, enriching the realm of LLM research and deployment with a vital dimension. D3LM surpasses classical LLMs by delivering outstanding performance and a remarkable user experience in the legal domain.