Representation Learning for Words and Entities

TL;DR

This thesis introduces new unsupervised methods for learning word and entity representations from text and knowledge bases, improving search, recommendation, and logical consistency in embeddings.

Contribution

It presents MVLSA for word embeddings, NVSE for entity representations, and novel approaches for knowledge graph embeddings respecting logical constraints.

Findings

MVLSA outperforms state-of-the-art word embedding models

NVSE improves search and recommendation in noisy data

Knowledge graph embeddings obey logical constraints

Abstract

This thesis presents new methods for unsupervised learning of distributed representations of words and entities from text and knowledge bases. The first algorithm presented in the thesis is a multi-view algorithm for learning representations of words called Multiview Latent Semantic Analysis (MVLSA). By incorporating up to 46 different types of co-occurrence statistics for the same vocabulary of english words, I show that MVLSA outperforms other state-of-the-art word embedding models. Next, I focus on learning entity representations for search and recommendation and present the second method of this thesis, Neural Variational Set Expansion (NVSE). NVSE is also an unsupervised learning method, but it is based on the Variational Autoencoder framework. Evaluations with human annotators show that NVSE can facilitate better search and recommendation of information gathered from noisy, automatic annotation of unstructured natural language corpora. Finally, I move from unstructured data and focus on structured knowledge graphs. I present novel approaches for learning embeddings of vertices and edges in a knowledge graph that obey logical constraints.