UniGen: Universal Domain Generalization for Sentiment Classification via Zero-shot Dataset Generation

TL;DR

This paper introduces UniGen, a universal dataset generation method that enables domain-generalized sentiment classification with small models, overcoming the limitations of large pre-trained models and domain-specific data generation.

Contribution

UniGen presents a novel approach for generating domain-agnostic datasets for sentiment classification, improving generalization across domains with significantly smaller models.

Findings

Achieves domain generalization in sentiment classification

Uses models orders of magnitude smaller than PLMs

Demonstrates effectiveness across various domains

Abstract

Although pre-trained language models have exhibited great flexibility and versatility with prompt-based few-shot learning, they suffer from the extensive parameter size and limited applicability for inference. Recent studies have suggested that PLMs be used as dataset generators and a tiny task-specific model be trained to achieve efficient inference. However, their applicability to various domains is limited because they tend to generate domain-specific datasets. In this work, we propose a novel approach to universal domain generalization that generates a dataset regardless of the target domain. This allows for generalization of the tiny task model to any domain that shares the label space, thus enhancing the real-world applicability of the dataset generation paradigm. Our experiments indicate that the proposed method accomplishes generalizability across various domains while using a parameter set that is orders of magnitude smaller than PLMs.