GO4Align: Group Optimization for Multi-Task Alignment

TL;DR

GO4Align introduces an adaptive multi-task optimization method that aligns task learning through dynamic grouping and risk-guided strategies, improving performance and efficiency across various benchmarks.

Contribution

It presents a novel multi-task optimization approach with adaptive group risk minimization, dynamically clustering tasks and leveraging risk information for better alignment.

Findings

Outperforms existing methods on multiple benchmarks.

Achieves higher accuracy with lower computational costs.

Effectively handles task imbalance through adaptive grouping.

Abstract

This paper proposes \textit{GO4Align}, a multi-task optimization approach that tackles task imbalance by explicitly aligning the optimization across tasks. To achieve this, we design an adaptive group risk minimization strategy, comprising two techniques in implementation: (i) dynamical group assignment, which clusters similar tasks based on task interactions; (ii) risk-guided group indicators, which exploit consistent task correlations with risk information from previous iterations. Comprehensive experimental results on diverse benchmarks demonstrate our method's performance superiority with even lower computational costs.