Similarity of Semantic Relations

TL;DR

This paper introduces Latent Relational Analysis (LRA), a novel method for measuring relational similarity between word pairs, improving accuracy over previous models and approaching human performance, with broad applications in NLP tasks.

Contribution

LRA extends the Vector Space Model by automatically deriving patterns, applying SVD smoothing, and incorporating synonyms, significantly enhancing relational similarity measurement.

Findings

LRA achieves 56% accuracy on analogy questions, close to human 57%.

LRA outperforms the previous VSM approach with 47% accuracy.

LRA improves semantic relation classification results.

Abstract

There are at least two kinds of similarity. Relational similarity is correspondence between relations, in contrast with attributional similarity, which is correspondence between attributes. When two words have a high degree of attributional similarity, we call them synonyms. When two pairs of words have a high degree of relational similarity, we say that their relations are analogous. For example, the word pair mason:stone is analogous to the pair carpenter:wood. This paper introduces Latent Relational Analysis (LRA), a method for measuring relational similarity. LRA has potential applications in many areas, including information extraction, word sense disambiguation, and information retrieval. Recently the Vector Space Model (VSM) of information retrieval has been adapted to measuring relational similarity, achieving a score of 47% on a collection of 374 college-level multiple-choice word analogy questions. In the VSM approach, the relation between a pair of words is characterized by a vector of frequencies of predefined patterns in a large corpus. LRA extends the VSM approach in three ways: (1) the patterns are derived automatically from the corpus, (2) the Singular Value Decomposition (SVD) is used to smooth the frequency data, and (3) automatically generated synonyms are used to explore variations of the word pairs. LRA achieves 56% on the 374 analogy questions, statistically equivalent to the average human score of 57%. On the related problem of classifying semantic relations, LRA achieves similar gains over the VSM.