Gradient Surgery for Multi-Task Learning

TL;DR

This paper introduces a gradient surgery method to mitigate conflicting gradients in multi-task learning, significantly improving efficiency and performance across various supervised and reinforcement learning tasks.

Contribution

It proposes a novel gradient projection technique to reduce gradient interference, enhancing multi-task learning effectiveness and compatibility with existing architectures.

Findings

Substantial performance improvements on multi-task benchmarks

Effective reduction of gradient conflicts in multi-task optimization

Model-agnostic approach compatible with various architectures

Abstract

While deep learning and deep reinforcement learning (RL) systems have demonstrated impressive results in domains such as image classification, game playing, and robotic control, data efficiency remains a major challenge. Multi-task learning has emerged as a promising approach for sharing structure across multiple tasks to enable more efficient learning. However, the multi-task setting presents a number of optimization challenges, making it difficult to realize large efficiency gains compared to learning tasks independently. The reasons why multi-task learning is so challenging compared to single-task learning are not fully understood. In this work, we identify a set of three conditions of the multi-task optimization landscape that cause detrimental gradient interference, and develop a simple yet general approach for avoiding such interference between task gradients. We propose a form of gradient surgery that projects a task's gradient onto the normal plane of the gradient of any other task that has a conflicting gradient. On a series of challenging multi-task supervised and multi-task RL problems, this approach leads to substantial gains in efficiency and performance. Further, it is model-agnostic and can be combined with previously-proposed multi-task architectures for enhanced performance.