A comprehensive framework for legal dispute analysis integrating prompt engineering and multi-dimensional knowledge graphs

TL;DR

This paper introduces a new framework combining prompt engineering and knowledge graphs to improve legal dispute analysis by large language models.

Contribution

A novel framework integrating prompt engineering and multi-dimensional knowledge graphs for legal dispute analysis is proposed.

Findings

The framework improved F1 scores from 0.356 to 0.714 in legal dispute analysis.

BLEU-4 and ROUGE-L F1 scores also showed significant improvements.

Legal professional content quality scores increased by 18-20 points.

Abstract

Legal dispute analysis is crucial for intelligent legal assistance systems. However, current Large Language Models (LLMs) face challenges in understanding complex legal concepts, maintaining reasoning consistency, and accurately citing legal sources. This study presents a framework combining prompt engineering with multi-dimensional knowledge graphs to improve LLM capabilities for legal dispute analysis. The framework comprises a three-stage hierarchical prompt structure (task definition, knowledge background, reasoning guidance) and a three-layer knowledge graph (legal classification ontology layer, representation layer, instance layer). Additionally, four supporting methods enable legal concept retrieval: direct code matching, semantic vector similarity, ontology path reasoning, and professional terminology matching. Systematic testing on 500 test samples integrated from six internationally recognized legal AI benchmark datasets demonstrates performance improvements for mainstream models: F1 score increased from 0.356 to 0.714, BLEU-4 reached 0.451, ROUGE-L F1 improved from 0.34 to 0.71, and legal professional content quality scores increased by 18-20 points (on a 100-point scale). This framework provides a technical approach for legal analysis, contributing to the advancement of intelligent legal assistance systems.

Figures