Unprejudiced Training Auxiliary Tasks Makes Primary Better: A Multi-Task Learning Perspective

TL;DR

This paper introduces an uncertainty-based multi-task learning method that balances training across tasks, improving primary task performance even with noisy auxiliary labels, and surpassing existing approaches.

Contribution

It proposes a novel impartial learning strategy that balances auxiliary and primary tasks using uncertainty, enhancing multi-task learning effectiveness.

Findings

Achieves comparable or better performance than state-of-the-art methods.

Effective in handling noisy auxiliary labels.

Robust across different auxiliary task settings.

Abstract

Human beings can leverage knowledge from relative tasks to improve learning on a primary task. Similarly, multi-task learning methods suggest using auxiliary tasks to enhance a neural network's performance on a specific primary task. However, previous methods often select auxiliary tasks carefully but treat them as secondary during training. The weights assigned to auxiliary losses are typically smaller than the primary loss weight, leading to insufficient training on auxiliary tasks and ultimately failing to support the main task effectively. To address this issue, we propose an uncertainty-based impartial learning method that ensures balanced training across all tasks. Additionally, we consider both gradients and uncertainty information during backpropagation to further improve performance on the primary task. Extensive experiments show that our method achieves performance comparable to or better than state-of-the-art approaches. Moreover, our weighting strategy is effective and robust in enhancing the performance of the primary task regardless the noise auxiliary tasks' pseudo labels.