GRAF: Graph Retrieval Augmented by Facts for Romanian Legal Multi-Choice Question Answering

TL;DR

This paper introduces GRAF, a novel graph retrieval method for Romanian legal multiple-choice question answering, supported by new datasets, a legal knowledge graph, and IR techniques, achieving state-of-the-art results in a low-resource language domain.

Contribution

It presents the first Romanian legal MCQA dataset, a legal knowledge graph, and a new graph retrieval approach that outperforms existing methods.

Findings

GRAF achieves competitive and superior results compared to SOTA methods.

The JuRO dataset enables research in low-resource legal NLP tasks.

The Law-RoG knowledge graph enhances legal question answering performance.

Abstract

Pre-trained Language Models (PLMs) have shown remarkable performances in recent years, setting a new paradigm for NLP research and industry. The legal domain has received some attention from the NLP community partly due to its textual nature. Some tasks from this domain are represented by question-answering (QA) tasks. This work explores the legal domain Multiple-Choice QA (MCQA) for a low-resource language. The contribution of this work is multi-fold. We first introduce JuRO, the first openly available Romanian legal MCQA dataset, comprising three different examinations and a number of 10,836 total questions. Along with this dataset, we introduce CROL, an organized corpus of laws that has a total of 93 distinct documents with their modifications from 763 time spans, that we leveraged in this work for Information Retrieval (IR) techniques. Moreover, we are the first to propose Law-RoG, a Knowledge Graph (KG) for the Romanian language, and this KG is derived from the aforementioned corpus. Lastly, we propose a novel approach for MCQA, Graph Retrieval Augmented by Facts (GRAF), which achieves competitive results with generally accepted SOTA methods and even exceeds them in most settings.