Online Active Regression

TL;DR

This paper introduces online algorithms for active regression that efficiently maintain linear models with minimal label queries, suitable for large and streaming datasets, and demonstrates their effectiveness both theoretically and empirically.

Contribution

The paper presents novel online active regression algorithms for $ ext{ell}_p$ loss that require fewer label queries and work efficiently in streaming settings, extending prior offline methods.

Findings

Algorithms achieve $(1+\epsilon)$-approximate solutions with $ ilde{O}(\epsilon^{-1} d \log(n\kappa))$ label queries.

Numerical experiments confirm theoretical bounds and show comparable performance to offline methods.

Methods are effective for large-scale, streaming data with limited label budgets.

Abstract

Active regression considers a linear regression problem where the learner receives a large number of data points but can only observe a small number of labels. Since online algorithms can deal with incremental training data and take advantage of low computational cost, we consider an online extension of the active regression problem: the learner receives data points one by one and immediately decides whether it should collect the corresponding labels. The goal is to efficiently maintain the regression of received data points with a small budget of label queries. We propose novel algorithms for this problem under $\ell_p$ loss where $p\in[1,2]$. To achieve a $(1+\epsilon)$-approximate solution, our proposed algorithms only require $\tilde{\mathcal{O}}(\epsilon^{-1} d \log(n\kappa))$ queries of labels, where $n$ is the number of data points and $\kappa$ is a quantity, called the condition number, of the data points. The numerical results verify our theoretical results and show that our methods have comparable performance with offline active regression algorithms.