Learning to Share by Masking the Non-shared for Multi-domain Sentiment Classification

TL;DR

This paper introduces BertMasker, a model that masks domain-related words to create domain-agnostic texts, improving multi-domain sentiment classification by explicitly distinguishing shared and private features.

Contribution

The paper proposes a novel masking-based approach to explicitly remove domain-related tokens, enhancing interpretability and domain-invariance in sentiment classification models.

Findings

BertMasker improves multi-domain sentiment classification accuracy by 0.94%.

It enhances cross-domain performance with a 1.8% accuracy increase.

Removing domain-related tokens reduces domain distinction, affecting classifier performance.

Abstract

Multi-domain sentiment classification deals with the scenario where labeled data exists for multiple domains but insufficient for training effective sentiment classifiers that work across domains. Thus, fully exploiting sentiment knowledge shared across domains is crucial for real world applications. While many existing works try to extract domain-invariant features in high-dimensional space, such models fail to explicitly distinguish between shared and private features at text-level, which to some extent lacks interpretablity. Based on the assumption that removing domain-related tokens from texts would help improve their domain-invariance, we instead first transform original sentences to be domain-agnostic. To this end, we propose the BertMasker network which explicitly masks domain-related words from texts, learns domain-invariant sentiment features from these domain-agnostic texts, and uses those masked words to form domain-aware sentence representations. Empirical experiments on a well-adopted multiple domain sentiment classification dataset demonstrate the effectiveness of our proposed model on both multi-domain sentiment classification and cross-domain settings, by increasing the accuracy by 0.94% and 1.8% respectively. Further analysis on masking proves that removing those domain-related and sentiment irrelevant tokens decreases texts' domain distinction, resulting in the performance degradation of a BERT-based domain classifier by over 12%.