Learning Taxonomies of Concepts and not Words using Contextualized Word Representations: A Position Paper

TL;DR

This paper proposes a novel approach to learning concept taxonomies by leveraging contextualized word representations, defining concepts as synsets, and measuring their semantic relationships, addressing limitations of word-based taxonomies.

Contribution

It introduces a new method that uses density-based approximations of contextualized representations to define and relate concepts as synsets, expanding taxonomy learning capabilities.

Findings

Proposes a density-based approach for concept representation

Defines concepts as synsets rather than single words

Enables measurement of similarity and hypernymy among concepts

Abstract

Taxonomies are semantic hierarchies of concepts. One limitation of current taxonomy learning systems is that they define concepts as single words. This position paper argues that contextualized word representations, which recently achieved state-of-the-art results on many competitive NLP tasks, are a promising method to address this limitation. We outline a novel approach for taxonomy learning that (1) defines concepts as synsets, (2) learns density-based approximations of contextualized word representations, and (3) can measure similarity and hypernymy among them.