Multi-task Bias-Variance Trade-off Through Functional Constraints

TL;DR

This paper introduces a novel multi-task learning approach that balances bias and variance through functional constraints, improving performance across multiple domains by enforcing solutions to stay close to a central function.

Contribution

It proposes a constrained learning framework with a stochastic primal-dual algorithm to control the bias-variance trade-off in multi-task learning.

Findings

Outperforms task-specific classifiers on real multi-domain data

Effectively balances bias and variance through functional constraints

Demonstrates improved generalization in multi-task classification

Abstract

Multi-task learning aims to acquire a set of functions, either regressors or classifiers, that perform well for diverse tasks. At its core, the idea behind multi-task learning is to exploit the intrinsic similarity across data sources to aid in the learning process for each individual domain. In this paper we draw intuition from the two extreme learning scenarios -- a single function for all tasks, and a task-specific function that ignores the other tasks dependencies -- to propose a bias-variance trade-off. To control the relationship between the variance (given by the number of i.i.d. samples), and the bias (coming from data from other task), we introduce a constrained learning formulation that enforces domain specific solutions to be close to a central function. This problem is solved in the dual domain, for which we propose a stochastic primal-dual algorithm. Experimental results for a multi-domain classification problem with real data show that the proposed procedure outperforms both the task specific, as well as the single classifiers.