Semi-supervised Vector-valued Learning: Improved Bounds and Algorithms

TL;DR

This paper develops sharper theoretical bounds and an efficient semi-supervised algorithm for vector-valued learning, enhancing performance in multi-task and transfer learning scenarios.

Contribution

It introduces novel semi-supervised excess risk bounds for vector-valued learning and proposes a new algorithm leveraging local Rademacher complexity and Laplacian regularization.

Findings

Sharper excess risk bounds than existing methods

Improved convergence rates based on total sample size

Proposed algorithm outperforms existing methods in experiments

Abstract

Vector-valued learning, where the output space admits a vector-valued structure, is an important problem that covers a broad family of important domains, e.g. multi-task learning and transfer learning. Using local Rademacher complexity and unlabeled data, we derive novel semi-supervised excess risk bounds for general vector-valued learning from both kernel perspective and linear perspective. The derived bounds are much sharper than existing ones and the convergence rates are improved from the square root of labeled sample size to the square root of total sample size or directly dependent on labeled sample size. Motivated by our theoretical analysis, we propose a general semi-supervised algorithm for efficiently learning vector-valued functions, incorporating both local Rademacher complexity and Laplacian regularization. Extensive experimental results illustrate the proposed algorithm significantly outperforms the compared methods, which coincides with our theoretical findings.