Transfer learning for piecewise-constant mean estimation: Optimality, $\ell_1$- and $\ell_0$-penalisation

TL;DR

This paper develops optimal transfer learning estimators for piecewise-constant signals using $$- and $$-penalties, incorporating source selection to improve accuracy across multiple, diverse data sources.

Contribution

It introduces new transfer learning estimators with $$- and $$-penalties, along with a source selection algorithm, achieving minimax optimality in multisource scenarios.

Findings

Estimators achieve minimax optimality in multisource transfer learning.

Higher observation frequencies and diverse source data improve estimation accuracy.

Extensive numerical experiments validate theoretical results.

Abstract

We study transfer learning for estimating piecewise-constant signals when source data, which may be relevant but disparate, are available in addition to the target data. We first investigate transfer learning estimators that respectively employ $\ell_1$- and $\ell_0$-penalties for unisource data scenarios and then generalise these estimators to accommodate multisources. To further reduce estimation errors, especially when some sources significantly differ from the target, we introduce an informative source selection algorithm. We then examine these estimators with multisource selection and establish their minimax optimality. Unlike the common narrative in the transfer learning literature that the performance is enhanced through large source sample sizes, our approaches leverage higher observation frequencies and accommodate diverse frequencies across multiple sources. Our theoretical findings are supported by extensive numerical experiments, with the code available online, see https://github.com/chrisfanwang/transferlearning