MaxGNR: A Dynamic Weight Strategy via Maximizing Gradient-to-Noise Ratio for Multi-Task Learning

TL;DR

This paper introduces MaxGNR, a novel dynamic weight strategy that maximizes the gradient-to-noise ratio to improve multi-task learning by mitigating inter-task gradient noise, leading to better performance on standard datasets.

Contribution

The paper proposes MaxGNR, a new algorithm that enhances multi-task learning by explicitly maximizing the gradient-to-noise ratio to address training inefficiencies.

Findings

MaxGNR outperforms baseline methods on NYUv2 and Cityscapes datasets.

Maximizing GNR reduces inter-task gradient noise interference.

Improved training stability and task performance in MTL.

Abstract

When modeling related tasks in computer vision, Multi-Task Learning (MTL) can outperform Single-Task Learning (STL) due to its ability to capture intrinsic relatedness among tasks. However, MTL may encounter the insufficient training problem, i.e., some tasks in MTL may encounter non-optimal situation compared with STL. A series of studies point out that too much gradient noise would lead to performance degradation in STL, however, in the MTL scenario, Inter-Task Gradient Noise (ITGN) is an additional source of gradient noise for each task, which can also affect the optimization process. In this paper, we point out ITGN as a key factor leading to the insufficient training problem. We define the Gradient-to-Noise Ratio (GNR) to measure the relative magnitude of gradient noise and design the MaxGNR algorithm to alleviate the ITGN interference of each task by maximizing the GNR of each task. We carefully evaluate our MaxGNR algorithm on two standard image MTL datasets: NYUv2 and Cityscapes. The results show that our algorithm outperforms the baselines under identical experimental conditions.