Stochastic Adversarial Networks for Multi-Domain Text Classification

TL;DR

This paper introduces Stochastic Adversarial Networks (SAN) for multi-domain text classification, which models domain-specific features as distributions to efficiently handle multiple domains without increasing model size.

Contribution

The paper proposes a novel SAN framework that models domain-specific parameters as Gaussian distributions, enabling scalable multi-domain learning with stable adversarial training.

Findings

SAN achieves competitive results on MDTC benchmarks.

Model size remains constant regardless of domain number.

Incorporates domain label smoothing and pseudo-label regularization.

Abstract

Adversarial training has been instrumental in advancing multi-domain text classification (MDTC). Traditionally, MDTC methods employ a shared-private paradigm, with a shared feature extractor for domain-invariant knowledge and individual private feature extractors for domain-specific knowledge. Despite achieving state-of-the-art results, these methods grapple with the escalating model parameters due to the continuous addition of new domains. To address this challenge, we introduce the Stochastic Adversarial Network (SAN), which innovatively models the parameters of the domain-specific feature extractor as a multivariate Gaussian distribution, as opposed to a traditional weight vector. This design allows for the generation of numerous domain-specific feature extractors without a substantial increase in model parameters, maintaining the model's size on par with that of a single domain-specific extractor. Furthermore, our approach integrates domain label smoothing and robust pseudo-label regularization to fortify the stability of adversarial training and to refine feature discriminability, respectively. The performance of our SAN, evaluated on two leading MDTC benchmarks, demonstrates its competitive edge against the current state-of-the-art methodologies. The code is available at https://github.com/wangxu0820/SAN.