Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction

TL;DR

This paper introduces a lightweight, fast domain adaptation method for deep networks that reconstructs early convolutional filters affected by domain shift, improving performance with limited target data.

Contribution

The paper proposes a novel, computationally efficient domain adaptation technique that reconstructs early convolutional filters to counteract domain shift effects.

Findings

Effective adaptation within minutes

Improved accuracy on benchmark datasets

Domain shift detected at early convolutional layers

Abstract

End-to-end learning methods have achieved impressive results in many areas of computer vision. At the same time, these methods still suffer from a degradation in performance when testing on new datasets that stem from a different distribution. This is known as the domain shift effect. Recently proposed adaptation methods focus on retraining the network parameters. However, this requires access to all (labeled) source data, a large amount of (unlabeled) target data, and plenty of computational resources. In this work, we propose a lightweight alternative, that allows adapting to the target domain based on a limited number of target samples in a matter of minutes rather than hours, days or even weeks. To this end, we first analyze the output of each convolutional layer from a domain adaptation perspective. Surprisingly, we find that already at the very first layer, domain shift effects pop up. We then propose a new domain adaptation method, where first layer convolutional filters that are badly affected by the domain shift are reconstructed based on less affected ones. This improves the performance of the deep network on various benchmark datasets.