Comparison bewteen multi-task and single-task oracle risks in kernel ridge regression

TL;DR

This paper analyzes when multi-task kernel ridge regression outperforms single-task methods by comparing oracle risks, providing explicit conditions and simulations to understand their relative performance.

Contribution

It offers explicit settings where multi-task oracle risk surpasses single-task risk and demonstrates the conditions under which multi-task methods are advantageous.

Findings

Multi-task oracle risk can be lower than single-task risk in certain settings.

Explicit conditions favoring multi-task performance are identified.

Simulations confirm theoretical predictions in natural scenarios.

Abstract

In this paper we study multi-task kernel ridge regression and try to understand when the multi-task procedure performs better than the single-task one, in terms of averaged quadratic risk. In order to do so, we compare the risks of the estimators with perfect calibration, the \emph{oracle risk}. We are able to give explicit settings, favorable to the multi-task procedure, where the multi-task oracle performs better than the single-task one. In situations where the multi-task procedure is conjectured to perform badly, we also show the oracle does so. We then complete our study with simulated examples, where we can compare both oracle risks in more natural situations. A consequence of our result is that the multi-task ridge estimator has a lower risk than any single-task estimator, in favorable situations.