Active Learning with Weak Supervision for Gaussian Processes

TL;DR

This paper introduces an active learning method for Gaussian Processes that optimizes both which data points to annotate and the precision of each annotation, balancing cost and model performance.

Contribution

It presents a novel acquisition function that incorporates annotation precision selection into active learning for Gaussian Processes, enhancing efficiency.

Findings

Adjusting annotation precision improves model performance under budget constraints

The proposed method outperforms standard active learning approaches

Empirical results demonstrate significant gains in learning efficiency

Abstract

Annotating data for supervised learning can be costly. When the annotation budget is limited, active learning can be used to select and annotate those observations that are likely to give the most gain in model performance. We propose an active learning algorithm that, in addition to selecting which observation to annotate, selects the precision of the annotation that is acquired. Assuming that annotations with low precision are cheaper to obtain, this allows the model to explore a larger part of the input space, with the same annotation budget. We build our acquisition function on the previously proposed BALD objective for Gaussian Processes, and empirically demonstrate the gains of being able to adjust the annotation precision in the active learning loop.