Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs

TL;DR

This paper introduces a novel low-rank multitask learning framework using tensorized SVMs and LSSVMs, effectively modeling complex task relations with high-order tensors and demonstrating superior performance over existing methods.

Contribution

It proposes a general tensor-based low-rank MTL approach with CP factorization, enabling better task relation modeling and applicability to classification and regression.

Findings

Outperforms state-of-the-art MTL methods in experiments

Models complex task relations via tensor CP factorization

Provides a weighted kernel function capturing task similarities

Abstract

Multitask learning (MTL) leverages task-relatedness to enhance performance. With the emergence of multimodal data, tasks can now be referenced by multiple indices. In this paper, we employ high-order tensors, with each mode corresponding to a task index, to naturally represent tasks referenced by multiple indices and preserve their structural relations. Based on this representation, we propose a general framework of low-rank MTL methods with tensorized support vector machines (SVMs) and least square support vector machines (LSSVMs), where the CP factorization is deployed over the coefficient tensor. Our approach allows to model the task relation through a linear combination of shared factors weighted by task-specific factors and is generalized to both classification and regression problems. Through the alternating optimization scheme and the Lagrangian function, each subproblem is transformed into a convex problem, formulated as a quadratic programming or linear system in the dual form. In contrast to previous MTL frameworks, our decision function in the dual induces a weighted kernel function with a task-coupling term characterized by the similarities of the task-specific factors, better revealing the explicit relations across tasks in MTL. Experimental results validate the effectiveness and superiority of our proposed methods compared to existing state-of-the-art approaches in MTL. The code of implementation will be available at https://github.com/liujiani0216/TSVM-MTL.