Hierarchies over Vector Space: Orienting Word and Graph Embeddings
Xingzhi Guo, Steven Skiena

TL;DR
This paper introduces a hierarchical data structure derived from flat embedding spaces that captures directional and hierarchical relationships among entities, improving tasks like hypernym detection and link recovery.
Contribution
The paper proposes a novel algorithm to construct hierarchical trees from unordered embeddings, leveraging entity power to reveal inherent hierarchies and directional relations.
Findings
Achieved 8.98% hypernym discovery accuracy across five languages.
Attained 62.76% accuracy in Wikipedia link recovery.
Demonstrated the effectiveness of hierarchy construction in multiple NLP tasks.
Abstract
Word and graph embeddings are widely used in deep learning applications. We present a data structure that captures inherent hierarchical properties from an unordered flat embedding space, particularly a sense of direction between pairs of entities. Inspired by the notion of \textit{distributional generality}, our algorithm constructs an arborescence (a directed rooted tree) by inserting nodes in descending order of entity power (e.g., word frequency), pointing each entity to the closest more powerful node as its parent. We evaluate the performance of the resulting tree structures on three tasks: hypernym relation discovery, least-common-ancestor (LCA) discovery among words, and Wikipedia page link recovery. We achieve average 8.98\% and 2.70\% for hypernym and LCA discovery across five languages and 62.76\% accuracy on directed Wiki-page link recovery, with both substantially above…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Biomedical Text Mining and Ontologies
