Miko Team: Deep Learning Approach for Legal Question Answering in ALQAC 2022

TL;DR

This paper presents a deep learning approach using XLM-RoBERTa for legal document retrieval and question answering, achieving top results in the ALQAC 2022 competition, especially effective with limited labeled data.

Contribution

The authors develop a pre-trained transformer-based method for legal QA that excels in low-resource settings and demonstrates state-of-the-art performance in a competitive environment.

Findings

Achieved 1st place in legal document retrieval

Achieved 3rd place in legal question answering

Effective in low-resource language settings

Abstract

We introduce efficient deep learning-based methods for legal document processing including Legal Document Retrieval and Legal Question Answering tasks in the Automated Legal Question Answering Competition (ALQAC 2022). In this competition, we achieve 1\textsuperscript{st} place in the first task and 3\textsuperscript{rd} place in the second task. Our method is based on the XLM-RoBERTa model that is pre-trained from a large amount of unlabeled corpus before fine-tuning to the specific tasks. The experimental results showed that our method works well in legal retrieval information tasks with limited labeled data. Besides, this method can be applied to other information retrieval tasks in low-resource languages.