Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extraction

TL;DR

This paper introduces Diff-KPE, a novel keyphrase extraction method that uses a variational information bottleneck-guided text diffusion process to generate and incorporate keyphrase embeddings, improving extraction accuracy.

Contribution

It proposes a new framework combining VIB and text diffusion for enhanced keyphrase representation and extraction, outperforming existing methods on benchmark datasets.

Findings

Outperforms existing KPE methods on OpenKP and KP20K datasets.

Effectively utilizes keyphrase and document information for ranking.

Demonstrates improved keyphrase extraction accuracy.

Abstract

Keyphrase extraction (KPE) is an important task in Natural Language Processing for many scenarios, which aims to extract keyphrases that are present in a given document. Many existing supervised methods treat KPE as sequential labeling, span-level classification, or generative tasks. However, these methods lack the ability to utilize keyphrase information, which may result in biased results. In this study, we propose Diff-KPE, which leverages the supervised Variational Information Bottleneck (VIB) to guide the text diffusion process for generating enhanced keyphrase representations. Diff-KPE first generates the desired keyphrase embeddings conditioned on the entire document and then injects the generated keyphrase embeddings into each phrase representation. A ranking network and VIB are then optimized together with rank loss and classification loss, respectively. This design of Diff-KPE allows us to rank each candidate phrase by utilizing both the information of keyphrases and the document. Experiments show that Diff-KPE outperforms existing KPE methods on a large open domain keyphrase extraction benchmark, OpenKP, and a scientific domain dataset, KP20K.