On the Discrimination Power and Effective Utilization of Active Learning Measures in Version Space Search

TL;DR

This paper analyzes various query selection measures in active learning, revealing their discrimination power, proposing improvements, and guiding their effective use in different AL scenarios.

Contribution

It provides a theoretical analysis of QSMs' discrimination power, introduces improved measures, and offers heuristic search methods for optimal queries in AL.

Findings

Certain QSMs outperform others in version space discrimination.

Improved QSMs address identified limitations of existing measures.

Guidelines for selecting QSMs in pool-based AL scenarios.

Abstract

Active Learning (AL) methods have proven cost-saving against passive supervised methods in many application domains. An active learner, aiming to find some target hypothesis, formulates sequential queries to some oracle. The set of hypotheses consistent with the already answered queries is called version space. Several query selection measures (QSMs) for determining the best query to ask next have been proposed. Assuming binaryoutcome queries, we analyze various QSMs wrt. to the discrimination power of their selected queries within the current version space. As a result, we derive superiority and equivalence relations between these QSMs and introduce improved versions of existing QSMs to overcome identified issues. The obtained picture gives a hint about which QSMs should preferably be used in pool-based AL scenarios. Moreover, we deduce properties optimal queries wrt. QSMs must satisfy. Based on these, we demonstrate how efficient heuristic search methods for optimal queries in query synthesis AL scenarios can be devised.