MetaKP: On-Demand Keyphrase Generation
Di Wu, Xiaoxian Shen, Kai-Wei Chang

TL;DR
MetaKP introduces on-demand keyphrase generation tailored to specific goals, utilizing a large benchmark and innovative prompting methods to improve performance and adaptability across domains.
Contribution
The paper presents MetaKP, a new benchmark and methods for on-demand keyphrase generation that adapt to user goals, including a novel self-consistency prompting approach with large language models.
Findings
Self-consistency prompting with GPT-4o outperforms fine-tuned models.
Performance drops under distribution shifts for supervised methods.
MetaKP enables applications like epidemic event detection.
Abstract
Traditional keyphrase prediction methods predict a single set of keyphrases per document, failing to cater to the diverse needs of users and downstream applications. To bridge the gap, we introduce on-demand keyphrase generation, a novel paradigm that requires keyphrases that conform to specific high-level goals or intents. For this task, we present MetaKP, a large-scale benchmark comprising four datasets, 7500 documents, and 3760 goals across news and biomedical domains with human-annotated keyphrases. Leveraging MetaKP, we design both supervised and unsupervised methods, including a multi-task fine-tuning approach and a self-consistency prompting method with large language models. The results highlight the challenges of supervised fine-tuning, whose performance is not robust to distribution shifts. By contrast, the proposed self-consistency prompting approach greatly improves the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsAdvanced Text Analysis Techniques
MethodsSparse Evolutionary Training
