Encoding Prior Knowledge with Eigenword Embeddings

TL;DR

This paper introduces a method to incorporate prior knowledge into canonical correlation analysis for deriving word embeddings, providing theoretical justification and empirical evaluation across multiple datasets.

Contribution

It presents a novel approach to integrate prior knowledge into CCA for word embedding generation, with theoretical backing and extensive testing.

Findings

Improved word embeddings with prior knowledge incorporation

Theoretical validation of the proposed method

Enhanced performance across diverse datasets

Abstract

Canonical correlation analysis (CCA) is a method for reducing the dimension of data represented using two views. It has been previously used to derive word embeddings, where one view indicates a word, and the other view indicates its context. We describe a way to incorporate prior knowledge into CCA, give a theoretical justification for it, and test it by deriving word embeddings and evaluating them on a myriad of datasets.