Semantic Vector Machines

TL;DR

This paper introduces a neural network architecture for learning semantic embeddings of words and sentences, improving NLP tasks like translation and sentence compression with comparable or better results than classical models.

Contribution

It presents a novel neural network framework for semantic embedding of sentences and words, enhancing NLP applications and demonstrating improved performance over traditional neural language models.

Findings

Effective embedding of n-grams for translation tasks

Comparable or improved sentence embedding results

A flexible neural architecture for semantic learning

Abstract

We first present our work in machine translation, during which we used aligned sentences to train a neural network to embed n-grams of different languages into an $d$-dimensional space, such that n-grams that are the translation of each other are close with respect to some metric. Good n-grams to n-grams translation results were achieved, but full sentences translation is still problematic. We realized that learning semantics of sentences and documents was the key for solving a lot of natural language processing problems, and thus moved to the second part of our work: sentence compression. We introduce a flexible neural network architecture for learning embeddings of words and sentences that extract their semantics, propose an efficient implementation in the Torch framework and present embedding results comparable to the ones obtained with classical neural language models, while being more powerful.