Combining keyphrase extraction and lexical diversity to characterize ideas in publication titles

TL;DR

This paper explores combining multiple keyphrase detection models with lexical diversity metrics to better characterize the evolution of ideas in scientific publication titles.

Contribution

It introduces a multi-model approach for keyphrase detection and analyzes how this improves the assessment of lexical diversity in research titles.

Findings

Multiple phrase detection models yield more comprehensive keyphrase sets.

Union and difference of keyphrase sets help identify non-specific phrases.

Lexical diversity metrics reflect shifts and expansion of research ideas.

Abstract

Beyond bibliometrics, there is interest in characterizing the evolution of the number of ideas in scientific papers. A common approach for investigating this involves analyzing the titles of publications to detect vocabulary changes over time. With the notion that phrases, or more specifically keyphrases, represent concepts, lexical diversity metrics are applied to phrased versions of the titles. Thus changes in lexical diversity are treated as indicators of shifts, and possibly expansion, of research. Therefore, optimizing detection of keyphrases is an important aspect of this process. Rather than just one, we propose to use multiple phrase detection models with the goal to produce a more comprehensive set of keyphrases from the source corpora. Another potential advantage to this approach is that the union and difference of these sets may provide automated techniques for identifying and omitting non-specific phrases. We compare the performance of several phrase detection models, analyze the keyphrase sets output of each, and calculate lexical diversity of corpora variants incorporating keyphrases from each model, using four common lexical diversity metrics.