What's in your Head? Emergent Behaviour in Multi-Task Transformer Models

TL;DR

This paper investigates how non-target heads in multi-task transformer models exhibit emergent behaviors that can explain or extend their trained tasks, revealing insights into model interpretability and generalization.

Contribution

It uncovers emergent behaviors in non-target heads of multi-task transformers, demonstrating their potential for interpretability and skill extrapolation.

Findings

Non-target heads show behaviors related to other tasks.

Emergent behaviors can explain or extend task capabilities.

Potential for improved interpretability and generalization.

Abstract

The primary paradigm for multi-task training in natural language processing is to represent the input with a shared pre-trained language model, and add a small, thin network (head) per task. Given an input, a target head is the head that is selected for outputting the final prediction. In this work, we examine the behaviour of non-target heads, that is, the output of heads when given input that belongs to a different task than the one they were trained for. We find that non-target heads exhibit emergent behaviour, which may either explain the target task, or generalize beyond their original task. For example, in a numerical reasoning task, a span extraction head extracts from the input the arguments to a computation that results in a number generated by a target generative head. In addition, a summarization head that is trained with a target question answering head, outputs query-based summaries when given a question and a context from which the answer is to be extracted. This emergent behaviour suggests that multi-task training leads to non-trivial extrapolation of skills, which can be harnessed for interpretability and generalization.