Learning to cluster in order to transfer across domains and tasks

TL;DR

This paper presents a new transfer learning method based on learning to cluster, which transfers similarity information to improve domain adaptation and cross-task clustering, achieving state-of-the-art results on multiple datasets.

Contribution

The paper introduces a novel approach that learns a similarity function for transfer learning, enabling effective domain adaptation and cross-task clustering without explicitly addressing domain discrepancy.

Findings

Achieved state-of-the-art results on Omniglot and ImageNet for cross-task clustering.

Demonstrated high accuracy in reconstructing semantic clusters.

Showed top accuracy in cross-domain transfer on Office-31 and SVHN-MNIST datasets.

Abstract

This paper introduces a novel method to perform transfer learning across domains and tasks, formulating it as a problem of learning to cluster. The key insight is that, in addition to features, we can transfer similarity information and this is sufficient to learn a similarity function and clustering network to perform both domain adaptation and cross-task transfer learning. We begin by reducing categorical information to pairwise constraints, which only considers whether two instances belong to the same class or not. This similarity is category-agnostic and can be learned from data in the source domain using a similarity network. We then present two novel approaches for performing transfer learning using this similarity function. First, for unsupervised domain adaptation, we design a new loss function to regularize classification with a constrained clustering loss, hence learning a clustering network with the transferred similarity metric generating the training inputs. Second, for cross-task learning (i.e., unsupervised clustering with unseen categories), we propose a framework to reconstruct and estimate the number of semantic clusters, again using the clustering network. Since the similarity network is noisy, the key is to use a robust clustering algorithm, and we show that our formulation is more robust than the alternative constrained and unconstrained clustering approaches. Using this method, we first show state of the art results for the challenging cross-task problem, applied on Omniglot and ImageNet. Our results show that we can reconstruct semantic clusters with high accuracy. We then evaluate the performance of cross-domain transfer using images from the Office-31 and SVHN-MNIST tasks and present top accuracy on both datasets. Our approach doesn't explicitly deal with domain discrepancy. If we combine with a domain adaptation loss, it shows further improvement.