Investigating Cross-Domain Behaviors of BERT in Review Understanding

TL;DR

This paper empirically studies how BERT models behave across different review domains, revealing that multi-domain fine-tuned models offer a good balance of performance and resource efficiency in review understanding tasks.

Contribution

It provides the first empirical analysis of cross-domain BERT behaviors in product review classification, comparing single-domain and multi-domain fine-tuning effects.

Findings

Multi-domain models outperform single-domain models on multi-domain data.

Single-domain models perform slightly better on their specific domain.

Multi-domain models reduce computational costs while maintaining competitive accuracy.

Abstract

Review score prediction requires review text understanding, a critical real-world application of natural language processing. Due to dissimilar text domains in product reviews, a common practice is fine-tuning BERT models upon reviews of differing domains. However, there has not yet been an empirical study of cross-domain behaviors of BERT models in the various tasks of product review understanding. In this project, we investigate text classification BERT models fine-tuned on single-domain and multi-domain Amazon review data. In our findings, though single-domain models achieved marginally improved performance on their corresponding domain compared to multi-domain models, multi-domain models outperformed single-domain models when evaluated on multi-domain data, single-domain data the single-domain model was not fine-tuned on, and on average when considering all tests. Though slight increases in accuracy can be achieved through single-domain model fine-tuning, computational resources and costs can be reduced by utilizing multi-domain models that perform well across domains.