Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback

TL;DR

This paper investigates the effectiveness of transformer-based models for German customer feedback classification, demonstrating that domain adaptation with unlabeled data significantly improves performance over off-the-shelf models.

Contribution

It introduces a domain adaptation approach for transformer models using unlabeled data, enhancing relevance and polarity classification in German customer feedback.

Findings

Transformer models outperform fastText baseline.

Domain adaptation improves model accuracy.

Achieved high F1-scores on GermEval 2017 tasks.

Abstract

Understanding customer feedback is becoming a necessity for companies to identify problems and improve their products and services. Text classification and sentiment analysis can play a major role in analyzing this data by using a variety of machine and deep learning approaches. In this work, different transformer-based models are utilized to explore how efficient these models are when working with a German customer feedback dataset. In addition, these pre-trained models are further analyzed to determine if adapting them to a specific domain using unlabeled data can yield better results than off-the-shelf pre-trained models. To evaluate the models, two downstream tasks from the GermEval 2017 are considered. The experimental results show that transformer-based models can reach significant improvements compared to a fastText baseline and outperform the published scores and previous models. For the subtask Relevance Classification, the best models achieve a micro-averaged $F1$-Score of 96.1 % on the first test set and 95.9 % on the second one, and a score of 85.1 % and 85.3 % for the subtask Polarity Classification.