Adaptive Sampling for Convex Regression

TL;DR

This paper presents a novel adaptive sampling method for convex function learning in the $L__$ norm, achieving near-optimal error rates and outperforming uniform sampling in experiments.

Contribution

It introduces the first principled adaptive-sampling procedure for convex regression, with a function-specific complexity measure and theoretical guarantees.

Findings

Method outperforms passive uniform sampling in experiments.

Nearly attains information-theoretic optimal error rates.

Suggests an idealized oracle strategy for adaptive sampling.

Abstract

In this paper, we introduce the first principled adaptive-sampling procedure for learning a convex function in the $L_\infty$ norm, a problem that arises often in the behavioral and social sciences. We present a function-specific measure of complexity and use it to prove that, for each convex function $f_{\star}$, our algorithm nearly attains the information-theoretically optimal, function-specific error rate. We also corroborate our theoretical contributions with numerical experiments, finding that our method substantially outperforms passive, uniform sampling for favorable synthetic and data-derived functions in low-noise settings with large sampling budgets. Our results also suggest an idealized "oracle strategy", which we use to gauge the potential advance of any adaptive-sampling strategy over passive sampling, for any given convex function.