Label Efficient Learning of Transferable Representations across Domains and Tasks

TL;DR

This paper introduces a label-efficient framework for learning transferable representations across domains and tasks, effectively handling domain shifts and limited labeled data, and demonstrating superior performance over fine-tuning in various transfer learning scenarios.

Contribution

The proposed method combines domain adversarial training with metric learning to enable effective transfer across domains and tasks with minimal labeled data, outperforming traditional fine-tuning approaches.

Findings

Outperforms fine-tuning on novel classes with few labels

Effective transfer from image recognition to video action recognition

Handles domain shift with domain adversarial loss

Abstract

We propose a framework that learns a representation transferable across different domains and tasks in a label efficient manner. Our approach battles domain shift with a domain adversarial loss, and generalizes the embedding to novel task using a metric learning-based approach. Our model is simultaneously optimized on labeled source data and unlabeled or sparsely labeled data in the target domain. Our method shows compelling results on novel classes within a new domain even when only a few labeled examples per class are available, outperforming the prevalent fine-tuning approach. In addition, we demonstrate the effectiveness of our framework on the transfer learning task from image object recognition to video action recognition.