Exploring Lightweight Large Language Models for Court View Generation

TL;DR

This paper systematically evaluates lightweight large language models for court view generation and charge prediction, revealing their potential in judicial AI with a new evaluation framework.

Contribution

It introduces CVGEvalKit, a comprehensive evaluation framework and datasets for assessing lightweight LLMs in legal AI tasks, and provides insights into their performance trade-offs.

Findings

Lightweight LLMs can effectively generate court views and predict charges.

Model architecture and size significantly influence performance.

Lightweight LLMs outperform DNNs in these legal tasks.

Abstract

Criminal Court View Generation (CVG) is a critical task in Legal Artificial Intelligence (Legal AI), involving the generation of court view based on case facts. In this work, we systematically explore the capabilities of lightweight (smaller than 2B) large language models (LLMs) in CVG and their impact on charge prediction. Our study addresses four key questions: (1) how does different architecture of LLMs affect the CVG quality and charge prediction. (2) how does LLMs size contribute to the performance, (3) how do lightweight LLMs compare with Deep Neural Networks (DNNs) in these tasks, and (4) how does predicting charge by court view generation first compare with predicting it directly. Additionally, we also develop CVGEvalKit, an evaluation framework including three public available datasets for CVG tasks, as well as predicting their charges. Comprehensive experiments are conducted on this framework, where models are trained on a mixed training set and evaluated on each dataset's test set. Experimental results provide new insights into the trade-offs between model architecture, model size, and the influence between different tasks, highlighting the potential of lightweight LLMs in judicial AI applications. The source code is anonymously available at \url{https://github.com/ZhitianHou/CVGEvalKit}