ISTRBoost: Importance Sampling Transfer Regression using Boosting

TL;DR

This paper introduces ISTRBoost, a boosting-based transfer learning method that uses importance sampling to reduce negative transfer and overfitting, showing improved and consistent performance across diverse datasets.

Contribution

The paper presents a novel importance sampling approach integrated with boosting for transfer regression, enhancing robustness and reducing negative transfer in transfer learning.

Findings

ISTRBoost outperforms competitors 63% of the time

It demonstrates consistent performance across diverse datasets

Reduces skewness caused by source dataset size

Abstract

Current Instance Transfer Learning (ITL) methodologies use domain adaptation and sub-space transformation to achieve successful transfer learning. However, these methodologies, in their processes, sometimes overfit on the target dataset or suffer from negative transfer if the test dataset has a high variance. Boosting methodologies have been shown to reduce the risk of overfitting by iteratively re-weighing instances with high-residual. However, this balance is usually achieved with parameter optimization, as well as reducing the skewness in weights produced due to the size of the source dataset. While the former can be achieved, the latter is more challenging and can lead to negative transfer. We introduce a simpler and more robust fix to this problem by building upon the popular boosting ITL regression methodology, two-stage TrAdaBoost.R2. Our methodology,~\us{}, is a boosting and random-forest based ensemble methodology that utilizes importance sampling to reduce the skewness due to the source dataset. We show that~\us{}~performs better than competitive transfer learning methodologies $63\%$ of the time. It also displays consistency in its performance over diverse datasets with varying complexities, as opposed to the sporadic results observed for other transfer learning methodologies.