Fusing Vector Space Models for Domain-Specific Applications

TL;DR

This paper introduces a method to automatically combine multiple domain-specific word embeddings, enhancing their expressiveness and improving machine learning performance in domain-specific applications.

Contribution

A novel approach that automatically selects and fuses multiple domain-specific embeddings using a ranking function and dimensionality reduction.

Findings

Improved performance on multiple domain-specific tasks.

Effective selection of relevant embeddings for specific domains.

Enhanced embedding compactness without losing expressiveness.

Abstract

We address the problem of tuning word embeddings for specific use cases and domains. We propose a new method that automatically combines multiple domain-specific embeddings, selected from a wide range of pre-trained domain-specific embeddings, to improve their combined expressive power. Our approach relies on two key components: 1) a ranking function, based on a new embedding similarity measure, that selects the most relevant embeddings to use given a domain and 2) a dimensionality reduction method that combines the selected embeddings to produce a more compact and efficient encoding that preserves the expressiveness. We empirically show that our method produces effective domain-specific embeddings that consistently improve the performance of state-of-the-art machine learning algorithms on multiple tasks, compared to generic embeddings trained on large text corpora.