On the sample complexity of semi-supervised multi-objective learning

TL;DR

This paper investigates the sample complexity in semi-supervised multi-objective learning, revealing conditions where unlabeled data can reduce labeled data needs, especially with Bregman losses.

Contribution

It provides theoretical bounds on sample complexity for MOL, highlighting when unlabeled data helps and introducing a semi-supervised pseudo-labeling algorithm.

Findings

Unavoidable complexity cost for some losses even with ideal semi-supervised data.

For Bregman losses, unlabeled data can significantly reduce labeled data requirements.

Proposed simple semi-supervised algorithm achieves the derived bounds.

Abstract

In multi-objective learning (MOL), several possibly competing prediction tasks must be solved jointly by a single model. Achieving good trade-offs may require a model class $\mathcal{G}$ with larger capacity than what is necessary for solving the individual tasks. This, in turn, increases the statistical cost, as reflected in known MOL bounds that depend on the complexity of $\mathcal{G}$. We show that this cost is unavoidable for some losses, even in an idealized semi-supervised setting, where the learner has access to the Bayes-optimal solutions for the individual tasks as well as the marginal distributions over the covariates. On the other hand, for objectives defined with Bregman losses, we prove that the complexity of $\mathcal{G}$ may come into play only in terms of unlabeled data. Concretely, we establish sample complexity upper bounds, showing precisely when and how unlabeled data can significantly alleviate the need for labeled data. These rates are achieved by a simple, semi-supervised algorithm via pseudo-labeling.