Measuring the relatedness between scientific publications using controlled vocabularies

TL;DR

This paper compares methods for measuring relatedness between scientific publications using controlled vocabularies, finding that soft cosine outperforms traditional cosine similarity in accuracy.

Contribution

Introduces two new methods, soft cosine and maximum term similarities, for better semantic relatedness measurement using controlled vocabularies.

Findings

Soft cosine is the most accurate method tested.

Traditional cosine similarity is less accurate than the new methods.

Results have implications for bibliometric analyses using controlled vocabularies.

Abstract

Measuring the relatedness between scientific publications is essential in many areas of bibliometrics and science policy. Controlled vocabularies provide a promising basis for measuring relatedness and are widely used in combination with Salton's cosine similarity. The latter is problematic because it only considers exact matches between terms. This article introduces two alternative methods - soft cosine and maximum term similarities - that account for the semantic similarity between non-matching terms. The article compares the accuracy of all three methods using the assignment of publications to topics in the TREC 2006 Genomics Track and the assumption that accurate relatedness measures should assign high relatedness scores to publication pairs within the same topic and low scores to pairs from separate topics. Results show that soft cosine is the most accurate method, while the most widely used version of Salton's cosine is markedly less accurate than the other methods tested. These findings have implications for how controlled vocabularies should be used to measure relatedness.