Sentence Embeddings for Russian NLU

TL;DR

This paper evaluates various sentence embedding models, including FastText, ELMo, and BERT, on multiple Russian language tasks using both supervised and unsupervised methods, and introduces new datasets for these tasks.

Contribution

It provides a comprehensive comparison of sentence embeddings for Russian NLP and introduces datasets for question answering and sentence prediction tasks.

Findings

BERT embeddings outperform FastText and ELMo on several tasks.

Supervised approaches yield better results than unsupervised methods.

New datasets facilitate future research in Russian sentence understanding.

Abstract

We investigate the performance of sentence embeddings models on several tasks for the Russian language. In our comparison, we include such tasks as multiple choice question answering, next sentence prediction, and paraphrase identification. We employ FastText embeddings as a baseline and compare it to ELMo and BERT embeddings. We conduct two series of experiments, using both unsupervised (i.e., based on similarity measure only) and supervised approaches for the tasks. Finally, we present datasets for multiple choice question answering and next sentence prediction in Russian.