Instance-based Inductive Deep Transfer Learning by Cross-Dataset Querying with Locality Sensitive Hashing

TL;DR

This paper introduces an inductive transfer learning method in NLP that enhances models by retrieving and integrating similar instances from different datasets using locality sensitive hashing, leading to improved performance and reduced data dependency.

Contribution

The paper presents a novel instance-based transfer learning approach that leverages cross-dataset instance retrieval with locality sensitive hashing without inheriting source model parameters.

Findings

Significant performance improvements on news classification datasets.

Reduced labeled data dependency for comparable accuracy.

Effective use of instance-level information for transfer learning.

Abstract

Supervised learning models are typically trained on a single dataset and the performance of these models rely heavily on the size of the dataset, i.e., amount of data available with the ground truth. Learning algorithms try to generalize solely based on the data that is presented with during the training. In this work, we propose an inductive transfer learning method that can augment learning models by infusing similar instances from different learning tasks in the Natural Language Processing (NLP) domain. We propose to use instance representations from a source dataset, \textit{without inheriting anything} from the source learning model. Representations of the instances of \textit{source} \& \textit{target} datasets are learned, retrieval of relevant source instances is performed using soft-attention mechanism and \textit{locality sensitive hashing}, and then, augmented into the model during training on the target dataset. Our approach simultaneously exploits the local \textit{instance level information} as well as the macro statistical viewpoint of the dataset. Using this approach we have shown significant improvements for three major news classification datasets over the baseline. Experimental evaluations also show that the proposed approach reduces dependency on labeled data by a significant margin for comparable performance. With our proposed cross dataset learning procedure we show that one can achieve competitive/better performance than learning from a single dataset.