Auxiliary Learning as an Asymmetric Bargaining Game

TL;DR

AuxiNash introduces a bargaining game framework to optimally balance auxiliary tasks in learning, improving generalization especially with small datasets, with proven convergence and superior benchmark performance.

Contribution

This work formalizes auxiliary task balancing as an asymmetric bargaining game and develops an efficient method to learn task bargaining power with theoretical guarantees.

Findings

AuxiNash outperforms existing methods on multiple benchmarks.

The proposed method converges with theoretical guarantees.

Task bargaining power correlates with contribution to main task performance.

Abstract

Auxiliary learning is an effective method for enhancing the generalization capabilities of trained models, particularly when dealing with small datasets. However, this approach may present several difficulties: (i) optimizing multiple objectives can be more challenging, and (ii) how to balance the auxiliary tasks to best assist the main task is unclear. In this work, we propose a novel approach, named AuxiNash, for balancing tasks in auxiliary learning by formalizing the problem as generalized bargaining game with asymmetric task bargaining power. Furthermore, we describe an efficient procedure for learning the bargaining power of tasks based on their contribution to the performance of the main task and derive theoretical guarantees for its convergence. Finally, we evaluate AuxiNash on multiple multi-task benchmarks and find that it consistently outperforms competing methods.