A Dictionary Approach to Domain-Invariant Learning in Deep Networks

TL;DR

This paper introduces a dictionary-based method for domain-invariant deep learning, decomposing convolutional layers into domain-specific atoms and shared coefficients to handle domain shifts efficiently with minimal additional parameters.

Contribution

The paper proposes a novel dictionary approach that explicitly models domain shifts in CNNs by decomposing layers into domain-specific atoms and shared coefficients, improving cross-domain performance.

Findings

Effective handling of domain shifts with few additional parameters.

Improved cross and joint domain performance demonstrated.

Theoretical and empirical validation across datasets and architectures.

Abstract

In this paper, we consider domain-invariant deep learning by explicitly modeling domain shifts with only a small amount of domain-specific parameters in a Convolutional Neural Network (CNN). By exploiting the observation that a convolutional filter can be well approximated as a linear combination of a small set of dictionary atoms, we show for the first time, both empirically and theoretically, that domain shifts can be effectively handled by decomposing a convolutional layer into a domain-specific atom layer and a domain-shared coefficient layer, while both remain convolutional. An input channel will now first convolve spatially only with each respective domain-specific dictionary atom to "absorb" domain variations, and then output channels are linearly combined using common decomposition coefficients trained to promote shared semantics across domains. We use toy examples, rigorous analysis, and real-world examples with diverse datasets and architectures, to show the proposed plug-in framework's effectiveness in cross and joint domain performance and domain adaptation. With the proposed architecture, we need only a small set of dictionary atoms to model each additional domain, which brings a negligible amount of additional parameters, typically a few hundred.