Multi-Faceted Hierarchical Multi-Task Learning for a Large Number of Tasks with Multi-dimensional Relations

TL;DR

This paper introduces a hierarchical multi-task learning model, MFH, designed to efficiently handle a large number of tasks with multi-dimensional relations, significantly improving performance in large-scale recommender systems, especially for new users.

Contribution

The paper proposes a novel Multi-Faceted Hierarchical MTL model that captures multi-dimensional task relations with a hierarchical structure, enhancing shared learning for large task sets.

Findings

MFH outperforms state-of-the-art MTL models in large-scale industry data.

MFH significantly improves online metrics for new users, including app time and retention.

The model effectively addresses cold-start problems in recommender systems.

Abstract

There has been many studies on improving the efficiency of shared learning in Multi-Task Learning(MTL). Previous work focused on the "micro" sharing perspective for a small number of tasks, while in Recommender Systems(RS) and other AI applications, there are often demands to model a large number of tasks with multi-dimensional task relations. For example, when using MTL to model various user behaviors in RS, if we differentiate new users and new items from old ones, there will be a cartesian product style increase of tasks with multi-dimensional relations. This work studies the "macro" perspective of shared learning network design and proposes a Multi-Faceted Hierarchical MTL model(MFH). MFH exploits the multi-dimension task relations with a nested hierarchical tree structure which maximizes the shared learning. We evaluate MFH and SOTA models in a large industry video platform of 10 billion samples and results show that MFH outperforms SOTA MTL models significantly in both offline and online evaluations across all user groups, especially remarkable for new users with an online increase of 9.1\% in app time per user and 1.85\% in next-day retention rate. MFH now has been deployed in a large scale online video recommender system. MFH is especially beneficial to the cold-start problems in RS where new users and new items often suffer from a "local overfitting" phenomenon. However, the idea is actually generic and widely applicable to other MTL scenarios.