Multinomial Adversarial Networks for Multi-Domain Text Classification

TL;DR

This paper introduces Multinomial Adversarial Networks (MANs), a theoretically grounded approach for multi-domain text classification that learns domain-invariant features, significantly improving performance especially in low-resource or unlabeled domain scenarios.

Contribution

The paper proposes a novel MAN framework that generalizes adversarial networks to multiple domains using f-divergence minimization, with theoretical justification and superior empirical results.

Findings

MAN outperforms prior methods on multi-domain text classification

MAN achieves state-of-the-art results in unlabeled domain settings

Theoretical analysis links MAN to f-divergence minimization

Abstract

Many text classification tasks are known to be highly domain-dependent. Unfortunately, the availability of training data can vary drastically across domains. Worse still, for some domains there may not be any annotated data at all. In this work, we propose a multinomial adversarial network (MAN) to tackle the text classification problem in this real-world multidomain setting (MDTC). We provide theoretical justifications for the MAN framework, proving that different instances of MANs are essentially minimizers of various f-divergence metrics (Ali and Silvey, 1966) among multiple probability distributions. MANs are thus a theoretically sound generalization of traditional adversarial networks that discriminate over two distributions. More specifically, for the MDTC task, MAN learns features that are invariant across multiple domains by resorting to its ability to reduce the divergence among the feature distributions of each domain. We present experimental results showing that MANs significantly outperform the prior art on the MDTC task. We also show that MANs achieve state-of-the-art performance for domains with no labeled data.