Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks

TL;DR

This paper investigates the fundamental limits of estimating a prior distribution over a VC class from multiple independent learning tasks, providing bounds and extensions relevant to transfer learning and economic applications.

Contribution

It derives upper and lower bounds on the convergence rates for prior estimation under smoothness conditions, extending results to real-valued functions and applications in economics.

Findings

Established optimal convergence rate bounds for prior estimation.

Proved consistency of the estimator for real-valued functions.

Discussed implications for transfer learning and preference elicitation.

Abstract

We study the optimal rates of convergence for estimating a prior distribution over a VC class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior. We specifically derive upper and lower bounds on the optimal rates under a smoothness condition on the correct prior, with the number of samples per data set equal the VC dimension. These results have implications for the improvements achievable via transfer learning. We additionally extend this setting to real-valued function, where we establish consistency of an estimator for the prior, and discuss an additional application to a preference elicitation problem in algorithmic economics.