Unsupervised Learning of Style-sensitive Word Vectors

TL;DR

This paper introduces an unsupervised method to learn style-sensitive word vectors by extending the CBOW model, supported by a new dataset and task for lexical stylistic similarity, demonstrating improved style representation.

Contribution

It proposes the first approach to capture stylistic similarity in word vectors using an extended CBOW model and introduces a new benchmark dataset for evaluating style-sensitive embeddings.

Findings

The extended CBOW model effectively captures stylistic nuances.

A new dataset supports the assumption that style is consistent within utterances.

Style-sensitive embeddings outperform baseline models in lexical stylistic similarity tasks.

Abstract

This paper presents the first study aimed at capturing stylistic similarity between words in an unsupervised manner. We propose extending the continuous bag of words (CBOW) model (Mikolov et al., 2013) to learn style-sensitive word vectors using a wider context window under the assumption that the style of all the words in an utterance is consistent. In addition, we introduce a novel task to predict lexical stylistic similarity and to create a benchmark dataset for this task. Our experiment with this dataset supports our assumption and demonstrates that the proposed extensions contribute to the acquisition of style-sensitive word embeddings.