Multi-Task Learning with Prior Information

TL;DR

This paper introduces a multi-task learning framework that leverages prior knowledge of feature relations, employing novel algorithms with proven convergence to improve generalization across related tasks.

Contribution

It proposes a new multi-task learning approach using prior feature relation knowledge and develops algorithms with linear convergence guarantees.

Findings

Algorithms improve generalization performance on real-world datasets.

The proposed methods outperform existing approaches in regression and classification tasks.

Theoretical guarantees ensure reliable convergence of the algorithms.

Abstract

Multi-task learning aims to boost the generalization performance of multiple related tasks simultaneously by leveraging information contained in those tasks. In this paper, we propose a multi-task learning framework, where we utilize prior knowledge about the relations between features. We also impose a penalty on the coefficients changing for each specific feature to ensure related tasks have similar coefficients on common features shared among them. In addition, we capture a common set of features via group sparsity. The objective is formulated as a non-smooth convex optimization problem, which can be solved with various methods, including gradient descent method with fixed stepsize, iterative shrinkage-thresholding algorithm (ISTA) with back-tracking, and its variation -- fast iterative shrinkage-thresholding algorithm (FISTA). In light of the sub-linear convergence rate of the methods aforementioned, we propose an asymptotically linear convergent algorithm with theoretical guarantee. Empirical experiments on both regression and classification tasks with real-world datasets demonstrate that our proposed algorithms are capable of improving the generalization performance of multiple related tasks.