Multi-Source Domain Adaptation meets Dataset Distillation through Dataset Dictionary Learning

TL;DR

This paper introduces MSDA-DD, a novel approach combining multi-source domain adaptation and dataset distillation, achieving state-of-the-art results with minimal data on various benchmarks.

Contribution

It proposes a new problem integrating MSDA and DD, adapting existing methods to synthesize compact datasets for effective domain adaptation.

Findings

State-of-the-art performance with as little as 1 sample per class.

Effective adaptation across diverse benchmarks.

Combines multiple techniques for dataset synthesis and domain transfer.

Abstract

In this paper, we consider the intersection of two problems in machine learning: Multi-Source Domain Adaptation (MSDA) and Dataset Distillation (DD). On the one hand, the first considers adapting multiple heterogeneous labeled source domains to an unlabeled target domain. On the other hand, the second attacks the problem of synthesizing a small summary containing all the information about the datasets. We thus consider a new problem called MSDA-DD. To solve it, we adapt previous works in the MSDA literature, such as Wasserstein Barycenter Transport and Dataset Dictionary Learning, as well as DD method Distribution Matching. We thoroughly experiment with this novel problem on four benchmarks (Caltech-Office 10, Tennessee-Eastman Process, Continuous Stirred Tank Reactor, and Case Western Reserve University), where we show that, even with as little as 1 sample per class, one achieves state-of-the-art adaptation performance.