Word Embeddings and Their Use In Sentence Classification Tasks

TL;DR

This paper explores word embeddings, their properties, and their combination with image embeddings, then demonstrates their effectiveness in sentence classification using a CNN trained on pre-trained vectors, achieving state-of-the-art results.

Contribution

It introduces a comprehensive discussion on word embeddings and empirically shows their superiority in sentence classification tasks with a CNN model.

Findings

Pre-trained word embeddings outperform random vectors in sentence classification.

Combining word and image embeddings enhances task performance.

CNN models trained on pre-trained embeddings achieve state-of-the-art results.

Abstract

This paper have two parts. In the first part we discuss word embeddings. We discuss the need for them, some of the methods to create them, and some of their interesting properties. We also compare them to image embeddings and see how word embedding and image embedding can be combined to perform different tasks. In the second part we implement a convolutional neural network trained on top of pre-trained word vectors. The network is used for several sentence-level classification tasks, and achieves state-of-art (or comparable) results, demonstrating the great power of pre-trainted word embeddings over random ones.