RATE: An LLM-Powered Retrieval Augmented Generation Technology-Extraction Pipeline

TL;DR

This paper presents RATE, a novel LLM-powered pipeline that combines retrieval augmented generation and validation to extract technology terms from scientific literature with high accuracy, demonstrated on BCI and XR research articles.

Contribution

The paper introduces RATE, a hybrid LLM-based pipeline that significantly improves technology extraction accuracy over existing methods, with broad applicability to scientific literature analysis.

Findings

RATE achieved an F1-score of 91.27%.

Outperformed BERT-based method with F1-score of 53.73%.

Mapped extracted technologies into a research landscape network.

Abstract

In an era of radical technology transformations, technology maps play a crucial role in enhancing decision making. These maps heavily rely on automated methods of technology extraction. This paper introduces Retrieval Augmented Technology Extraction (RATE), a Large Language Model (LLM) based pipeline for automated technology extraction from scientific literature. RATE combines Retrieval Augmented Generation (RAG) with multi-definition LLM-based validation. This hybrid method results in high recall in candidate generation alongside with high precision in candidate filtering. While the pipeline is designed to be general and widely applicable, we demonstrate its use on 678 research articles focused on Brain-Computer Interfaces (BCIs) and Extended Reality (XR) as a case study. Consequently, The validated technology terms by RATE were mapped into a co-occurrence network, revealing thematic clusters and structural features of the research landscape. For the purpose of evaluation, a gold standard dataset of technologies in 70 selected random articles had been curated by the experts. In addition, a technology extraction model based on Bidirectional Encoder Representations of Transformers (BERT) was used as a comparative method. RATE achieved F1-score of 91.27%, Significantly outperforming BERT with F1-score of 53.73%. Our findings highlight the promise of definition-driven LLM methods for technology extraction and mapping. They also offer new insights into emerging trends within the BCI-XR field. The source code is available https://github.com/AryaAftab/RATE