Distributionally Robust Active Learning for Gaussian Process Regression

TL;DR

This paper introduces two distributionally robust active learning methods for Gaussian process regression that aim to minimize worst-case expected errors across potential target distributions, with theoretical guarantees and empirical validation.

Contribution

It proposes novel active learning algorithms for GPR that address distributional uncertainty and provide theoretical bounds on worst-case errors.

Findings

The methods effectively reduce worst-case expected errors.

Theoretical bounds show errors can be made arbitrarily small.

Empirical results validate the methods on synthetic and real datasets.

Abstract

Gaussian process regression (GPR) or kernel ridge regression is a widely used and powerful tool for nonlinear prediction. Therefore, active learning (AL) for GPR, which actively collects data labels to achieve an accurate prediction with fewer data labels, is an important problem. However, existing AL methods do not theoretically guarantee prediction accuracy for target distribution. Furthermore, as discussed in the distributionally robust learning literature, specifying the target distribution is often difficult. Thus, this paper proposes two AL methods that effectively reduce the worst-case expected error for GPR, which is the worst-case expectation in target distribution candidates. We show an upper bound of the worst-case expected squared error, which suggests that the error will be arbitrarily small by a finite number of data labels under mild conditions. Finally, we demonstrate the effectiveness of the proposed methods through synthetic and real-world datasets.