GradCraft: Elevating Multi-task Recommendations through Holistic Gradient Crafting

TL;DR

GradCraft is a novel gradient crafting method that improves multi-task recommendation systems by balancing gradient magnitudes and directions, leading to better overall performance.

Contribution

It introduces a new approach to multi-task learning in recommendations by dynamically balancing gradient magnitudes and directions, addressing limitations of existing methods.

Findings

Enhanced multi-task recommendation performance

Effective gradient conflict resolution

Validated through offline and online experiments

Abstract

Recommender systems require the simultaneous optimization of multiple objectives to accurately model user interests, necessitating the application of multi-task learning methods. However, existing multi-task learning methods in recommendations overlook the specific characteristics of recommendation scenarios, falling short in achieving proper gradient balance. To address this challenge, we set the target of multi-task learning as attaining the appropriate magnitude balance and the global direction balance, and propose an innovative methodology named GradCraft in response. GradCraft dynamically adjusts gradient magnitudes to align with the maximum gradient norm, mitigating interference from gradient magnitudes for subsequent manipulation. It then employs projections to eliminate gradient conflicts in directions while considering all conflicting tasks simultaneously, theoretically guaranteeing the global resolution of direction conflicts. GradCraft ensures the concurrent achievement of appropriate magnitude balance and global direction balance, aligning with the inherent characteristics of recommendation scenarios. Both offline and online experiments attest to the efficacy of GradCraft in enhancing multi-task performance in recommendations. The source code for GradCraft can be accessed at https://github.com/baiyimeng/GradCraft.