Artificial Intuition: Efficient Classification of Scientific Abstracts

TL;DR

This paper introduces a novel workflow leveraging Large Language Models to classify short scientific abstracts into coarse, domain-specific categories, aiding strategic research insights despite the challenges posed by brevity and lack of context.

Contribution

It presents a new method that uses LLMs to generate metadata for classifying scientific abstracts, addressing a gap in automated categorization of short texts.

Findings

LLMs can effectively generate metadata for classification

The workflow improves classification accuracy for short abstracts

New assessment tools complement existing performance metrics

Abstract

It is desirable to coarsely classify short scientific texts, such as grant or publication abstracts, for strategic insight or research portfolio management. These texts efficiently transmit dense information to experts possessing a rich body of knowledge to aid interpretation. Yet this task is remarkably difficult to automate because of brevity and the absence of context. To address this gap, we have developed a novel approach to generate and appropriately assign coarse domain-specific labels. We show that a Large Language Model (LLM) can provide metadata essential to the task, in a process akin to the augmentation of supplemental knowledge representing human intuition, and propose a workflow. As a pilot study, we use a corpus of award abstracts from the National Aeronautics and Space Administration (NASA). We develop new assessment tools in concert with established performance metrics.