THUIR@COLIEE 2023: Incorporating Structural Knowledge into Pre-trained Language Models for Legal Case Retrieval

TL;DR

This paper presents a structure-aware pre-trained language model approach for legal case retrieval, incorporating heuristic processing and learning-to-rank techniques, achieving top performance in the COLIEE 2023 competition.

Contribution

It introduces a novel structure-aware pre-trained model combined with heuristic and learning-to-rank methods for improved legal case retrieval.

Findings

Our method outperforms all other submissions in COLIEE 2023.

The structure-aware model enhances understanding of legal texts.

Heuristic processing reduces irrelevant message influence.

Abstract

Legal case retrieval techniques play an essential role in modern intelligent legal systems. As an annually well-known international competition, COLIEE is aiming to achieve the state-of-the-art retrieval model for legal texts. This paper summarizes the approach of the championship team THUIR in COLIEE 2023. To be specific, we design structure-aware pre-trained language models to enhance the understanding of legal cases. Furthermore, we propose heuristic pre-processing and post-processing approaches to reduce the influence of irrelevant messages. In the end, learning-to-rank methods are employed to merge features with different dimensions. Experimental results demonstrate the superiority of our proposal. Official results show that our run has the best performance among all submissions. The implementation of our method can be found at https://github.com/CSHaitao/THUIR-COLIEE2023.