Domain Representative Keywords Selection: A Probabilistic Approach

TL;DR

This paper introduces a probabilistic method for selecting representative keywords that distinguish a target domain from a context domain, improving keyword summarization and trending keyword detection in NLP tasks.

Contribution

It presents a novel two-component mixture model and an efficient optimization algorithm for selecting distinctive, representative keywords with proven near-optimal approximation guarantees.

Findings

Outperforms baseline methods in keyword summary tasks

Effective in trending keywords selection across multiple domains

Demonstrates computational efficiency and high accuracy

Abstract

We propose a probabilistic approach to select a subset of a \textit{target domain representative keywords} from a candidate set, contrasting with a context domain. Such a task is crucial for many downstream tasks in natural language processing. To contrast the target domain and the context domain, we adapt the \textit{two-component mixture model} concept to generate a distribution of candidate keywords. It provides more importance to the \textit{distinctive} keywords of the target domain than common keywords contrasting with the context domain. To support the \textit{representativeness} of the selected keywords towards the target domain, we introduce an \textit{optimization algorithm} for selecting the subset from the generated candidate distribution. We have shown that the optimization algorithm can be efficiently implemented with a near-optimal approximation guarantee. Finally, extensive experiments on multiple domains demonstrate the superiority of our approach over other baselines for the tasks of keyword summary generation and trending keywords selection.