Transfer Learning for Improving Results on Russian Sentiment Datasets

TL;DR

This paper explores transfer learning techniques to enhance sentiment analysis on Russian datasets, achieving over 3% improvement and reaching human-level performance on one dataset.

Contribution

It introduces a three-step transfer learning approach and demonstrates its effectiveness in improving sentiment classification results on Russian datasets.

Findings

Over 3% improvement over state-of-the-art methods

BERT-NLI model reaches human-level performance on one dataset

Sequential training on general, thematic, and original data enhances results

Abstract

In this study, we test transfer learning approach on Russian sentiment benchmark datasets using additional train sample created with distant supervision technique. We compare several variants of combining additional data with benchmark train samples. The best results were achieved using three-step approach of sequential training on general, thematic and original train samples. For most datasets, the results were improved by more than 3% to the current state-of-the-art methods. The BERT-NLI model treating sentiment classification problem as a natural language inference task reached the human level of sentiment analysis on one of the datasets.