Learning Rates for Multi-task Regularization Networks

TL;DR

This paper provides a mathematical analysis of learning rates in multi-task regularization networks, revealing how the number of tasks influences generalization ability, based on vector-valued RKHS theory.

Contribution

It introduces an explicit learning rate estimate for multi-task regularization networks, connecting sample size and task number, which was previously underexplored.

Findings

Learning rate depends on sample size and number of tasks.

Generalization ability decreases as the number of tasks increases.

Provides theoretical foundation for multi-task learning analysis.

Abstract

Multi-task learning is an important trend of machine learning in facing the era of artificial intelligence and big data. Despite a large amount of researches on learning rate estimates of various single-task machine learning algorithms, there is little parallel work for multi-task learning. We present mathematical analysis on the learning rate estimate of multi-task learning based on the theory of vector-valued reproducing kernel Hilbert spaces and matrix-valued reproducing kernels. For the typical multi-task regularization networks, an explicit learning rate dependent both on the number of sample data and the number of tasks is obtained. It reveals that the generalization ability of multi-task learning algorithms is indeed affected as the number of tasks increases.