GAIus: Combining Genai with Legal Clauses Retrieval for Knowledge-based Assistant

TL;DR

This paper introduces gAIus, a legal knowledge-based assistant combining large language models with a retrieval system for Polish legal texts, significantly improving answer accuracy and explainability.

Contribution

The paper presents a novel retrieval mechanism integrated with LLMs for legal tasks, outperforming embedding-based methods and enhancing answer reliability.

Findings

Achieved a 419% improvement over GPT-3.5-turbo-0125.

Outperformed GPT-4o and GPT-4o-mini on legal question dataset.

Demonstrated better explainability and human-friendliness in legal information retrieval.

Abstract

In this paper we discuss the capability of large language models to base their answer and provide proper references when dealing with legal matters of non-english and non-chinese speaking country. We discuss the history of legal information retrieval, the difference between case law and statute law, its impact on the legal tasks and analyze the latest research in this field. Basing on that background we introduce gAIus, the architecture of the cognitive LLM-based agent, whose responses are based on the knowledge retrieved from certain legal act, which is Polish Civil Code. We propose a retrieval mechanism which is more explainable, human-friendly and achieves better results than embedding-based approaches. To evaluate our method we create special dataset based on single-choice questions from entrance exams for law apprenticeships conducted in Poland. The proposed architecture critically leveraged the abilities of used large language models, improving the gpt-3.5-turbo-0125 by 419%, allowing it to beat gpt-4o and lifting gpt-4o-mini score from 31% to 86%. At the end of our paper we show the possible future path of research and potential applications of our findings.