Deep Mutual Learning across Task Towers for Effective Multi-Task Recommender Learning

TL;DR

This paper introduces a novel deep mutual learning framework across task towers in multi-task recommender systems, enhancing knowledge sharing and improving prediction accuracy through a flexible architecture.

Contribution

It proposes a new architecture for multi-task learning in recommender systems that promotes better knowledge sharing across task towers, outperforming traditional methods.

Findings

Improved prediction accuracy in offline experiments

Enhanced online performance in AB tests

Flexible framework compatible with various networks

Abstract

Recommender systems usually leverage multi-task learning methods to simultaneously optimize several objectives because of the multi-faceted user behavior data. The typical way of conducting multi-task learning is to establish appropriate parameter sharing across multiple tasks at lower layers while reserving a separate task tower for each task at upper layers. Since the task towers exert direct impact on the prediction results, we argue that the architecture of standalone task towers is sub-optimal for promoting positive knowledge sharing. Accordingly, we propose the framework of Deep Mutual Learning across task towers, which is compatible with various backbone multi-task networks. Extensive offline experiments and online AB tests are conducted to evaluate and verify the proposed approach's effectiveness.