Do LLMs Know What Luxembourgish Borrows? Probing Lexical Neology in Low-Resource Multilingual Models

TL;DR

This paper introduces LexNeo-Bench, a benchmark for evaluating multilingual LLMs on lexical borrowing in Luxembourgish, demonstrating that knowledge-graph prompts significantly improve borrowing classification accuracy.

Contribution

It presents a novel benchmark and demonstrates that lexicon-aware prompting with structured lexical resources enhances LLM performance on borrowing detection in low-resource languages.

Findings

Knowledge-graph prompts increase borrowing classification accuracy from 25-35% to 71-81%.

Prompting largely closes the performance gap between small and large models.

Lexicon-aware prompting improves robustness in low-resource multilingual contexts.

Abstract

Large language models (LLMs) are increasingly used for writing assistance in small contact languages, yet it is unclear whether they respect community norms around lexical borrowing and neology. We introduce LexNeo-Bench, a 3{,}050-instance token-level benchmark derived from LuxBorrow, a large-scale Luxembourgish news corpus, where target tokens are labelled as native or as French, German, or English borrowings. Using this benchmark, we probe three multilingual LLMs across 34 prompt settings on two tasks: borrowing type classification and a binary lexical-innovation proxy (borrowing versus native). Without external context, models perform only slightly above chance on borrowing classification, so we construct a linguistic knowledge graph that encodes donor language, morphological patterns, and lexical analogues, and inject instance-specific subgraphs into the prompt. Knowledge-graph prompts raise borrowing classification accuracy from 25 -- 35\% up to 71 -- 81\% and largely close the gap between small and large models, while leaving neology detection difficult and sensitive to few-shot design. Our results show that lexicon-aware prompting is highly beneficial for robust borrowing judgments in low-resource contact languages and that lexical resources can serve as structured context for LLM evaluation. This study was carried out within the ENEOLI COST Action and examines borrowing as a form of lexical innovation in multilingual Luxembourgish data.