ERU-KG: Efficient Reference-aligned Unsupervised Keyphrase Generation

TL;DR

ERU-KG is an unsupervised keyphrase generation model that improves informativeness estimation using reference-based modeling, achieving high accuracy and efficiency, and can adapt for generation or extraction tasks.

Contribution

The paper introduces ERU-KG, a novel unsupervised keyphrase generation approach that models informativeness through references and term-level aggregation, enhancing accuracy and speed.

Findings

Outperforms unsupervised baselines on keyphrase benchmarks.

Achieves 89% of supervised model performance for top 10 keyphrases.

Faster inference speed compared to similar models.

Abstract

Unsupervised keyphrase prediction has gained growing interest in recent years. However, existing methods typically rely on heuristically defined importance scores, which may lead to inaccurate informativeness estimation. In addition, they lack consideration for time efficiency. To solve these problems, we propose ERU-KG, an unsupervised keyphrase generation (UKG) model that consists of an informativeness and a phraseness module. The former estimates the relevance of keyphrase candidates, while the latter generate those candidates. The informativeness module innovates by learning to model informativeness through references (e.g., queries, citation contexts, and titles) and at the term-level, thereby 1) capturing how the key concepts of documents are perceived in different contexts and 2) estimating informativeness of phrases more efficiently by aggregating term informativeness, removing the need for explicit modeling of the candidates. ERU-KG demonstrates its effectiveness on keyphrase generation benchmarks by outperforming unsupervised baselines and achieving on average 89\% of the performance of a supervised model for top 10 predictions. Additionally, to highlight its practical utility, we evaluate the model on text retrieval tasks and show that keyphrases generated by ERU-KG are effective when employed as query and document expansions. Furthermore, inference speed tests reveal that ERU-KG is the fastest among baselines of similar model sizes. Finally, our proposed model can switch between keyphrase generation and extraction by adjusting hyperparameters, catering to diverse application requirements.