Learning Output Kernels for Multi-Task Problems

TL;DR

This paper introduces a kernel-based multi-task learning method that automatically uncovers task relationships and jointly learns multiple functions, improving performance on pharmacological and collaborative filtering datasets.

Contribution

We propose a novel output kernel learning approach that jointly learns task relationships and multiple functions through a low-rank kernel, optimized with block coordinate descent.

Findings

Effective in revealing inter-task relationships

Improves multi-task learning performance

Validated on pharmacological and collaborative filtering data

Abstract

Simultaneously solving multiple related learning tasks is beneficial under a variety of circumstances, but the prior knowledge necessary to correctly model task relationships is rarely available in practice. In this paper, we develop a novel kernel-based multi-task learning technique that automatically reveals structural inter-task relationships. Building over the framework of output kernel learning (OKL), we introduce a method that jointly learns multiple functions and a low-rank multi-task kernel by solving a non-convex regularization problem. Optimization is carried out via a block coordinate descent strategy, where each subproblem is solved using suitable conjugate gradient (CG) type iterative methods for linear operator equations. The effectiveness of the proposed approach is demonstrated on pharmacological and collaborative filtering data.