A Batch Normalization Classifier for Domain Adaptation

TL;DR

This paper introduces a simple yet effective domain adaptation technique using batch normalization in the output layer of a ResNet model, which improves generalization across visual data domains with minimal computational cost.

Contribution

The novel contribution is applying batch normalization at the output layer for domain adaptation, outperforming many existing methods with negligible added complexity.

Findings

Outperforms many domain adaptation methods on Office-Home dataset.

Not sensitive to presence of source data during adaptation.

Leads to sparser tensor distributions after training.

Abstract

Adapting a model to perform well on unforeseen data outside its training set is a common problem that continues to motivate new approaches. We demonstrate that application of batch normalization in the output layer, prior to softmax activation, results in improved generalization across visual data domains in a refined ResNet model. The approach adds negligible computational complexity yet outperforms many domain adaptation methods that explicitly learn to align data domains. We benchmark this technique on the Office-Home dataset and show that batch normalization is competitive with other leading methods. We show that this method is not sensitive to presence of source data during adaptation and further present the impact on trained tensor distributions tends toward sparsity. Code is available at https://github.com/matthewbehrend/BNC