Active Learning by Query by Committee with Robust Divergences

TL;DR

This paper introduces a robust active learning method using Bregman divergences, including the $eta$-divergence and dual $\gamma$-power divergence, which outperform traditional Kullback-Leibler divergence-based approaches.

Contribution

The paper proposes a novel active learning query by committee method utilizing Bregman divergences for improved robustness against outliers.

Findings

The proposed method is more robust than traditional KL divergence-based methods.

Experimental results show comparable or better performance of the new method.

The approach generalizes the disagreement measure in active learning to a broader class of divergences.

Abstract

Active learning is a widely used methodology for various problems with high measurement costs. In active learning, the next object to be measured is selected by an acquisition function, and measurements are performed sequentially. The query by committee is a well-known acquisition function. In conventional methods, committee disagreement is quantified by the Kullback--Leibler divergence. In this paper, the measure of disagreement is defined by the Bregman divergence, which includes the Kullback--Leibler divergence as an instance, and the dual $\gamma$-power divergence. As a particular class of the Bregman divergence, the $\beta$-divergence is considered. By deriving the influence function, we show that the proposed method using $\beta$-divergence and dual $\gamma$-power divergence are more robust than the conventional method in which the measure of disagreement is defined by the Kullback--Leibler divergence. Experimental results show that the proposed method performs as well as or better than the conventional method.