Near-optimal Active Regression of Single-Index Models

TL;DR

This paper introduces a near-optimal active regression algorithm for single-index models that achieves a $(1+ ext{epsilon})$-approximation with query complexity nearly matching the theoretical lower bounds, improving over previous constant-factor methods.

Contribution

It presents the first algorithm for active regression of single-index models that attains a $(1+ ext{epsilon})$-approximation with nearly optimal query complexity, advancing beyond prior constant-factor approximations.

Findings

Achieves $(1+\varepsilon)$-approximation with $ ilde{O}(d^{p/2 \vee 1}/\varepsilon^{p \vee 2})$ queries.

Query complexity is optimal up to logarithmic factors for $p\in [1,2]$.

The $1/\varepsilon^p$ dependence is proven to be optimal for $p>2$.

Abstract

The active regression problem of the single-index model is to solve $\min_x \lVert f(Ax)-b\rVert_p$, where $A$ is fully accessible and $b$ can only be accessed via entry queries, with the goal of minimizing the number of queries to the entries of $b$. When $f$ is Lipschitz, previous results only obtain constant-factor approximations. This work presents the first algorithm that provides a $(1+\varepsilon)$-approximation solution by querying $\tilde{O}(d^{\frac{p}{2}\vee 1}/\varepsilon^{p\vee 2})$ entries of $b$. This query complexity is also shown to be optimal up to logarithmic factors for $p\in [1,2]$ and the $\varepsilon$-dependence of $1/\varepsilon^p$ is shown to be optimal for $p>2$.