Named entity recognition for Serbian legal documents: Design, methodology and dataset development

TL;DR

This paper presents a novel BERT-based Named Entity Recognition system tailored for Serbian legal documents, including dataset creation, system design, and evaluation, achieving high accuracy and robustness for legal text processing.

Contribution

The work introduces a new dataset and a specialized BERT-based NER model for Serbian legal documents, advancing NLP tools in this domain.

Findings

Achieved mean F1 score of 0.96 on the dataset

Demonstrated robustness through modified text input tests

Provided a comprehensive system design and methodology

Abstract

Recent advancements in the field of natural language processing (NLP) and especially large language models (LLMs) and their numerous applications have brought research attention to design of different document processing tools and enhancements in the process of document archiving, search and retrieval. Domain of official, legal documents is especially interesting due to vast amount of data generated on the daily basis, as well as the significant community of interested practitioners (lawyers, law offices, administrative workers, state institutions and citizens). Providing efficient ways for automation of everyday work involving legal documents is therefore expected to have significant impact in different fields. In this work we present one LLM based solution for Named Entity Recognition (NER) in the case of legal documents written in Serbian language. It leverages on the pre-trained bidirectional encoder representations from transformers (BERT), which had been carefully adapted to the specific task of identifying and classifying specific data points from textual content. Besides novel dataset development for Serbian language (involving public court rulings), presented system design and applied methodology, the paper also discusses achieved performance metrics and their implications for objective assessment of the proposed solution. Performed cross-validation tests on the created manually labeled dataset with mean $F_1$ score of 0.96 and additional results on the examples of intentionally modified text inputs confirm applicability of the proposed system design and robustness of the developed NER solution.