An Exploration of Data Efficiency in Intra-Dataset Task Transfer for Dialog Understanding

TL;DR

This paper investigates how transfer learning impacts data efficiency in dialog understanding, revealing that target data size often minimally affects transfer performance, possibly due to catastrophic forgetting.

Contribution

It provides an empirical analysis of transfer learning effects in dialog tasks, highlighting the limited impact of target data size and suggesting new directions to address catastrophic forgetting.

Findings

Target data size often has minimal effect on transfer learning performance.

Catastrophic forgetting may hinder transfer learning effectiveness.

Further research needed to develop methods preventing forgetting.

Abstract

Transfer learning is an exciting area of Natural Language Processing that has the potential to both improve model performance and increase data efficiency. This study explores the effects of varying quantities of target task training data on sequential transfer learning in the dialog domain. We hypothesize that a model can utilize the information learned from a source task to better learn a target task, thereby reducing the number of target task training samples required. Unintuitively, our data shows that often target task training data size has minimal effect on how sequential transfer learning performs compared to the same model without transfer learning. Our results lead us to believe that this unexpected result could be due to the effects of catastrophic forgetting, motivating further work into methods that prevent such forgetting.