SEMIE: SEMantically Infused Embeddings with Enhanced Interpretability for Domain-specific Small Corpus

TL;DR

This paper introduces SEMIE, a method for creating highly interpretable and efficient word embeddings tailored for small, domain-specific corpora, addressing limitations of generic embeddings in specialized fields.

Contribution

The paper proposes a novel approach to generate interpretable embeddings specifically designed for small, domain-specific datasets, enhancing their applicability in specialized NLP tasks.

Findings

Embeddings demonstrate improved interpretability in domain contexts

Enhanced efficiency of embeddings for small corpora

Evaluation shows competitive performance with existing methods

Abstract

Word embeddings are a basic building block of modern NLP pipelines. Efforts have been made to learn rich, efficient, and interpretable embeddings for large generic datasets available in the public domain. However, these embeddings have limited applicability for small corpora from specific domains such as automotive, manufacturing, maintenance and support, etc. In this work, we present a comprehensive notion of interpretability for word embeddings and propose a novel method to generate highly interpretable and efficient embeddings for a domain-specific small corpus. We report the evaluation results of our resulting word embeddings and demonstrate their novel features for enhanced interpretability.