Chernoff Sampling for Active Testing and Extension to Active Regression

TL;DR

This paper revisits Chernoff's asymptotically optimal hypothesis testing algorithm, providing new non-asymptotic bounds, and extends it to active parameter estimation for neural networks and regression, showing improved performance.

Contribution

It introduces a novel non-asymptotic analysis of Chernoff's algorithm and extends it for active parameter estimation in diverse models, including neural networks and regression.

Findings

Provides a non-asymptotic sample complexity bound for Chernoff's test.

Develops an extension for active parameter estimation with error bounds.

Demonstrates improved active learning performance on neural networks and regression tasks.

Abstract

Active learning can reduce the number of samples needed to perform a hypothesis test and to estimate the parameters of a model. In this paper, we revisit the work of Chernoff that described an asymptotically optimal algorithm for performing a hypothesis test. We obtain a novel sample complexity bound for Chernoff's algorithm, with a non-asymptotic term that characterizes its performance at a fixed confidence level. We also develop an extension of Chernoff sampling that can be used to estimate the parameters of a wide variety of models and we obtain a non-asymptotic bound on the estimation error. We apply our extension of Chernoff sampling to actively learn neural network models and to estimate parameters in real-data linear and non-linear regression problems, where our approach performs favorably to state-of-the-art methods.