Investigating Expert-in-the-Loop LLM Discourse Patterns for Ancient Intertextual Analysis

TL;DR

This paper evaluates large language models for detecting and analyzing intertextual relationships in biblical and Greek texts, highlighting their capabilities and limitations with expert-in-the-loop methodology.

Contribution

It introduces an expert-in-the-loop approach for using LLMs to analyze intertextuality in ancient texts, demonstrating its scalability and potential for new insights.

Findings

LLMs can identify direct quotations, allusions, and echoes.

Models generate novel intertextual connections.

Challenges include handling long passages and false dependencies.

Abstract

This study explores the potential of large language models (LLMs) for identifying and examining intertextual relationships within biblical, Koine Greek texts. By evaluating the performance of LLMs on various intertextuality scenarios the study demonstrates that these models can detect direct quotations, allusions, and echoes between texts. The LLM's ability to generate novel intertextual observations and connections highlights its potential to uncover new insights. However, the model also struggles with long query passages and the inclusion of false intertextual dependences, emphasizing the importance of expert evaluation. The expert-in-the-loop methodology presented offers a scalable approach for intertextual research into the complex web of intertextuality within and beyond the biblical corpus.