Multi-task Highly Adaptive Lasso

TL;DR

The paper introduces MT-HAL, a nonparametric multi-task learning method that automatically discovers shared sparse structures among tasks, achieving fast convergence and outperforming existing methods in diverse simulations.

Contribution

It presents a novel, fully nonparametric multi-task learning approach that learns feature, sample, and task associations with a shared sparse structure, along with a fast convergence rate.

Findings

Outperforms existing sparsity-based MTL methods in simulations

Effective in both linear and nonlinear settings

Achieves a convergence rate of o_p(n^{-1/4}) or better

Abstract

We propose a novel, fully nonparametric approach for the multi-task learning, the Multi-task Highly Adaptive Lasso (MT-HAL). MT-HAL simultaneously learns features, samples and task associations important for the common model, while imposing a shared sparse structure among similar tasks. Given multiple tasks, our approach automatically finds a sparse sharing structure. The proposed MTL algorithm attains a powerful dimension-free convergence rate of $o_p(n^{-1/4})$ or better. We show that MT-HAL outperforms sparsity-based MTL competitors across a wide range of simulation studies, including settings with nonlinear and linear relationships, varying levels of sparsity and task correlations, and different numbers of covariates and sample size.