Finding Needles in Emb(a)dding Haystacks: Legal Document Retrieval via Bagging and SVR Ensembles

TL;DR

This paper presents a novel ensemble-based retrieval method using Support Vector Regression and embedding spaces for legal document retrieval, achieving improved recall without deep learning models.

Contribution

It introduces a bagging and SVR ensemble approach for legal document retrieval that outperforms baseline methods in recall, without requiring deep learning training.

Findings

Recall improved to 0.849 with ensemble

Effective in binary needle-in-a-haystack tasks

No deep learning training or fine-tuning needed

Abstract

We introduce a retrieval approach leveraging Support Vector Regression (SVR) ensembles, bootstrap aggregation (bagging), and embedding spaces on the German Dataset for Legal Information Retrieval (GerDaLIR). By conceptualizing the retrieval task in terms of multiple binary needle-in-a-haystack subtasks, we show improved recall over the baselines (0.849 > 0.803 | 0.829) using our voting ensemble, suggesting promising initial results, without training or fine-tuning any deep learning models. Our approach holds potential for further enhancement, particularly through refining the encoding models and optimizing hyperparameters.