Toward Improving Attentive Neural Networks in Legal Text Processing

TL;DR

This paper reviews challenges and improvements in applying attentive neural networks to legal text processing, highlighting the need for domain-specific adaptation and expertise to handle complex legal language.

Contribution

It presents key advancements in enhancing attentive neural networks for legal document processing and discusses the limitations of large language models without domain expertise.

Findings

Legal sentences are long and contain complex terminology.

Standard models struggle with legal domain adaptation.

Experiments verify the challenges in legal NLP tasks.

Abstract

In recent years, thanks to breakthroughs in neural network techniques especially attentive deep learning models, natural language processing has made many impressive achievements. However, automated legal word processing is still a difficult branch of natural language processing. Legal sentences are often long and contain complicated legal terminologies. Hence, models that work well on general documents still face challenges in dealing with legal documents. We have verified the existence of this problem with our experiments in this work. In this dissertation, we selectively present the main achievements in improving attentive neural networks in automatic legal document processing. Language models tend to grow larger and larger, though, without expert knowledge, these models can still fail in domain adaptation, especially for specialized fields like law.