Multi-task Learning for Heterogeneous Data via Integrating Shared and Task-Specific Encodings

TL;DR

This paper introduces a dual-encoder multi-task learning framework that effectively captures shared and task-specific information, addressing heterogeneity challenges and improving predictive performance across diverse applications.

Contribution

It proposes a novel dual-encoder approach with a unified algorithm to handle distribution and posterior heterogeneity in multi-task learning.

Findings

Outperforms existing data integration methods in simulations

Achieves superior prediction of tumor doubling time across cancer types

Provides theoretical excess risk bounds for the proposed method

Abstract

Multi-task learning (MTL) has become an essential machine learning tool for addressing multiple learning tasks simultaneously and has been effectively applied across fields such as healthcare, marketing, and biomedical research. However, to enable efficient information sharing across tasks, it is crucial to leverage both shared and heterogeneous information. Despite extensive research on MTL, various forms of heterogeneity, including distribution and posterior heterogeneity, present significant challenges. Existing methods often fail to address these forms of heterogeneity within a unified framework. In this paper, we propose a dual-encoder framework to construct a heterogeneous latent factor space for each task, incorporating a task-shared encoder to capture common information across tasks and a task-specific encoder to preserve unique task characteristics. Additionally, we explore the intrinsic similarity structure of the coefficients corresponding to learned latent factors, allowing for adaptive integration across tasks to manage posterior heterogeneity. We introduce a unified algorithm that alternately learns the task-specific and task-shared encoders and coefficients. In theory, we investigate the excess risk bound for the proposed MTL method using local Rademacher complexity and apply it to a new but related task. Through simulation studies, we demonstrate that the proposed method outperforms existing data integration methods across various settings. Furthermore, the proposed method achieves superior predictive performance for time to tumor doubling across five distinct cancer types in PDX data.