Domain Adaptation for Neural Networks by Parameter Augmentation

TL;DR

This paper introduces a straightforward supervised domain adaptation method for neural networks that enhances performance on target domains by augmenting model parameters, demonstrated on NLP captioning tasks.

Contribution

It presents a novel parameter augmentation approach for neural network domain adaptation, improving upon existing methods in supervised settings.

Findings

Performance improvements over existing domain adaptation techniques.

Effective on NLP captioning datasets.

Applicable to neural networks trained with cross-entropy loss.

Abstract

We propose a simple domain adaptation method for neural networks in a supervised setting. Supervised domain adaptation is a way of improving the generalization performance on the target domain by using the source domain dataset, assuming that both of the datasets are labeled. Recently, recurrent neural networks have been shown to be successful on a variety of NLP tasks such as caption generation; however, the existing domain adaptation techniques are limited to (1) tune the model parameters by the target dataset after the training by the source dataset, or (2) design the network to have dual output, one for the source domain and the other for the target domain. Reformulating the idea of the domain adaptation technique proposed by Daume (2007), we propose a simple domain adaptation method, which can be applied to neural networks trained with a cross-entropy loss. On captioning datasets, we show performance improvements over other domain adaptation methods.