Taking into Account the Differences between Actively and Passively Acquired Data: The Case of Active Learning with Support Vector Machines for Imbalanced Datasets

TL;DR

This paper explores how active learning with cost-weighted SVMs can better handle imbalanced datasets by estimating overall corpus imbalance from small unbiased samples, improving over traditional passive learning methods.

Contribution

It introduces the InitPA method that adjusts cost models during active learning based on corpus imbalance estimates, addressing limitations of passive learning approaches.

Findings

InitPA improves imbalance handling in active learning.

Active learning with InitPA outperforms passive learning in imbalanced scenarios.

The method is effective for high-imbalance datasets in HLT tasks.

Abstract

Actively sampled data can have very different characteristics than passively sampled data. Therefore, it's promising to investigate using different inference procedures during AL than are used during passive learning (PL). This general idea is explored in detail for the focused case of AL with cost-weighted SVMs for imbalanced data, a situation that arises for many HLT tasks. The key idea behind the proposed InitPA method for addressing imbalance is to base cost models during AL on an estimate of overall corpus imbalance computed via a small unbiased sample rather than the imbalance in the labeled training data, which is the leading method used during PL.