On the Feasibility of Automated Detection of Allusive Text Reuse

TL;DR

This paper explores the feasibility of automatically detecting allusive text reuse by leveraging lexical semantics and information retrieval techniques, highlighting challenges and potential improvements in retrieval accuracy.

Contribution

It introduces a novel approach combining lexical semantic information with IR methods and provides an inter-annotator agreement study for benchmark corpus creation.

Findings

Manual queries improve retrieval over windowing methods

Distributional semantics moderately boost retrieval performance

Low inter-annotator agreement highlights annotation challenges

Abstract

The detection of allusive text reuse is particularly challenging due to the sparse evidence on which allusive references rely---commonly based on none or very few shared words. Arguably, lexical semantics can be resorted to since uncovering semantic relations between words has the potential to increase the support underlying the allusion and alleviate the lexical sparsity. A further obstacle is the lack of evaluation benchmark corpora, largely due to the highly interpretative character of the annotation process. In the present paper, we aim to elucidate the feasibility of automated allusion detection. We approach the matter from an Information Retrieval perspective in which referencing texts act as queries and referenced texts as relevant documents to be retrieved, and estimate the difficulty of benchmark corpus compilation by a novel inter-annotator agreement study on query segmentation. Furthermore, we investigate to what extent the integration of lexical semantic information derived from distributional models and ontologies can aid retrieving cases of allusive reuse. The results show that (i) despite low agreement scores, using manual queries considerably improves retrieval performance with respect to a windowing approach, and that (ii) retrieval performance can be moderately boosted with distributional semantics.