Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrases

TL;DR

This paper evaluates the effectiveness of prompt-based large language models in generating Russian scientific keyphrases, comparing various methods and analyzing their strengths and weaknesses through human expert assessments.

Contribution

It introduces the application of prompt-based LLMs for Russian keyphrase generation and compares their performance with traditional methods.

Findings

Prompt-based methods outperform baselines in keyphrase generation.

Few-shot prompt strategies improve model performance.

Human evaluation confirms the quality of generated keyphrases.

Abstract

Keyphrase selection is a challenging task in natural language processing that has a wide range of applications. Adapting existing supervised and unsupervised solutions for the Russian language faces several limitations due to the rich morphology of Russian and the limited number of training datasets available. Recent studies conducted on English texts show that large language models (LLMs) successfully address the task of generating keyphrases. LLMs allow achieving impressive results without task-specific fine-tuning, using text prompts instead. In this work, we access the performance of prompt-based methods for generating keyphrases for Russian scientific abstracts. First, we compare the performance of zero-shot and few-shot prompt-based methods, fine-tuned models, and unsupervised methods. Then we assess strategies for selecting keyphrase examples in a few-shot setting. We present the outcomes of human evaluation of the generated keyphrases and analyze the strengths and weaknesses of the models through expert assessment. Our results suggest that prompt-based methods can outperform common baselines even using simple text prompts.