Modelling Semantic Categories using Conceptual Neighborhood

TL;DR

This paper introduces a method to improve semantic category modeling in vector spaces by leveraging the concept of conceptual neighbors, which are closely related but distinct categories that are spatially adjacent in the embedding space.

Contribution

The paper proposes a novel approach that uses conceptual neighborhood information to enhance the accuracy of category region representations in embedding spaces.

Findings

Incorporating conceptual neighbors improves category region estimation.

The method effectively captures interdependent category relationships.

Results show more accurate semantic region modeling in embeddings.

Abstract

While many methods for learning vector space embeddings have been proposed in the field of Natural Language Processing, these methods typically do not distinguish between categories and individuals. Intuitively, if individuals are represented as vectors, we can think of categories as (soft) regions in the embedding space. Unfortunately, meaningful regions can be difficult to estimate, especially since we often have few examples of individuals that belong to a given category. To address this issue, we rely on the fact that different categories are often highly interdependent. In particular, categories often have conceptual neighbors, which are disjoint from but closely related to the given category (e.g.\ fruit and vegetable). Our hypothesis is that more accurate category representations can be learned by relying on the assumption that the regions representing such conceptual neighbors should be adjacent in the embedding space. We propose a simple method for identifying conceptual neighbors and then show that incorporating these conceptual neighbors indeed leads to more accurate region based representations.