The Multiverse Loss for Robust Transfer Learning

TL;DR

This paper introduces a multiverse loss approach that trains multiple orthogonal classifiers in the source domain, leading to more discriminative and robust transfer learning representations, as validated on CIFAR-100 and LFW datasets.

Contribution

It proposes learning multiple orthogonal classifiers in the source domain to enhance the discriminative capacity of transfer learning representations.

Findings

Supports more discriminative directions in the feature space

Produces similar softmax probabilities across classifiers

Improves transfer learning performance on CIFAR-100 and LFW

Abstract

Deep learning techniques are renowned for supporting effective transfer learning. However, as we demonstrate, the transferred representations support only a few modes of separation and much of its dimensionality is unutilized. In this work, we suggest to learn, in the source domain, multiple orthogonal classifiers. We prove that this leads to a reduced rank representation, which, however, supports more discriminative directions. Interestingly, the softmax probabilities produced by the multiple classifiers are likely to be identical. Experimental results, on CIFAR-100 and LFW, further demonstrate the effectiveness of our method.