Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval

TL;DR

This paper introduces a syntactic retrieval prompting strategy for large language models that enhances automatic term extraction performance across domains by focusing on syntactic similarity, leading to improved F1-scores.

Contribution

It presents a novel syntactic retrieval method for LLM prompting in ATE, demonstrating its effectiveness in both in-domain and cross-domain scenarios.

Findings

Syntactic retrieval improves F1-score in ATE tasks.

Lexical overlap influences retrieval effectiveness.

Syntactic cues are crucial for adapting LLMs to terminology extraction.

Abstract

Automatic Term Extraction (ATE) identifies domain-specific expressions that are crucial for downstream tasks such as machine translation and information retrieval. Although large language models (LLMs) have significantly advanced various NLP tasks, their potential for ATE has scarcely been examined. We propose a retrieval-based prompting strategy that, in the few-shot setting, selects demonstrations according to \emph{syntactic} rather than semantic similarity. This syntactic retrieval method is domain-agnostic and provides more reliable guidance for capturing term boundaries. We evaluate the approach in both in-domain and cross-domain settings, analyzing how lexical overlap between the query sentence and its retrieved examples affects performance. Experiments on three specialized ATE benchmarks show that syntactic retrieval improves F1-score. These findings highlight the importance of syntactic cues when adapting LLMs to terminology-extraction tasks.