Friend-training: Learning from Models of Different but Related Tasks

TL;DR

Friend-training introduces a cross-task self-training framework where models trained on related tasks iteratively teach each other, improving performance in dialogue understanding tasks beyond traditional single-task methods.

Contribution

This work proposes a novel cross-task self-training framework called friend-training, leveraging related tasks to enhance model performance through mutual pseudo-labeling and retraining.

Findings

Models trained with friend-training outperform strong baselines.

The framework effectively utilizes related tasks for mutual improvement.

Case study on dialogue understanding tasks demonstrates significant gains.

Abstract

Current self-training methods such as standard self-training, co-training, tri-training, and others often focus on improving model performance on a single task, utilizing differences in input features, model architectures, and training processes. However, many tasks in natural language processing are about different but related aspects of language, and models trained for one task can be great teachers for other related tasks. In this work, we propose friend-training, a cross-task self-training framework, where models trained to do different tasks are used in an iterative training, pseudo-labeling, and retraining process to help each other for better selection of pseudo-labels. With two dialogue understanding tasks, conversational semantic role labeling and dialogue rewriting, chosen for a case study, we show that the models trained with the friend-training framework achieve the best performance compared to strong baselines.