# Transductive Auxiliary Task Self-Training for Neural Multi-Task Models

## TL;DR

This paper introduces transductive auxiliary task self-training, a method that enhances multi-task models by leveraging test instances with auxiliary labels, leading to significant accuracy improvements in dependency and semantic tagging tasks.

## Contribution

The paper proposes a novel transductive self-training approach that combines multi-task learning with test-time auxiliary labels to improve model performance.

## Key findings

- Up to 9.56% accuracy improvement in dependency relation tagging.
- Up to 13.03% accuracy improvement in semantic tagging.
- Extensive experiments on 86 language-task combinations demonstrate effectiveness.

## Abstract

Multi-task learning and self-training are two common ways to improve a machine learning model's performance in settings with limited training data. Drawing heavily on ideas from those two approaches, we suggest transductive auxiliary task self-training: training a multi-task model on (i) a combination of main and auxiliary task training data, and (ii) test instances with auxiliary task labels which a single-task version of the model has previously generated. We perform extensive experiments on 86 combinations of languages and tasks. Our results are that, on average, transductive auxiliary task self-training improves absolute accuracy by up to 9.56% over the pure multi-task model for dependency relation tagging and by up to 13.03% for semantic tagging.