Transfer Regression via Pairwise Similarity Regularization

TL;DR

This paper introduces a transfer learning approach based on pairwise similarity regularization, allowing knowledge transfer even when source and target functions differ, by leveraging shared pairwise prediction patterns.

Contribution

It proposes a novel graph-based regularization framework that encodes pairwise similarity assumptions, enabling transfer learning beyond identical source and target functions.

Findings

Effective on real and synthetic datasets

Can incorporate domain knowledge via similarity constraints

Scalable with Nystrom approximation

Abstract

Transfer learning methods address the situation where little labeled training data from the "target" problem exists, but much training data from a related "source" domain is available. However, the overwhelming majority of transfer learning methods are designed for simple settings where the source and target predictive functions are almost identical, limiting the applicability of transfer learning methods to real world data. We propose a novel, weaker, property of the source domain that can be transferred even when the source and target predictive functions diverge. Our method assumes the source and target functions share a Pairwise Similarity property, where if the source function makes similar predictions on a pair of instances, then so will the target function. We propose Pairwise Similarity Regularization Transfer, a flexible graph-based regularization framework which can incorporate this modeling assumption into standard supervised learning algorithms. We show how users can encode domain knowledge into our regularizer in the form of spatial continuity, pairwise "similarity constraints" and how our method can be scaled to large data sets using the Nystrom approximation. Finally, we present positive and negative results on real and synthetic data sets and discuss when our Pairwise Similarity transfer assumption seems to hold in practice.