On the Evaluation Criterions for the Active Learning Processes

TL;DR

This paper explores evaluation criteria for active learning, demonstrating that competitive results can be achieved with limited data and proposing alternative evaluation methods to improve AL process assessment.

Contribution

It introduces new evaluation criteria for active learning and shows that effective results are possible with small data subsets.

Findings

Competitive results achieved with small data percentages

Proposed alternative evaluation criteria for AL processes

Discussion of AL Challenge results at WCCI 2010

Abstract

In many data mining applications collection of sufficiently large datasets is the most time consuming and expensive. On the other hand, industrial methods of data collection create huge databases, and make difficult direct applications of the advanced machine learning algorithms. To address the above problems, we consider active learning (AL), which may be very efficient either for the experimental design or for the data filtering. In this paper we demonstrate using the online evaluation opportunity provided by the AL Challenge that quite competitive results may be produced using a small percentage of the available data. Also, we present several alternative criteria, which may be useful for the evaluation of the active learning processes. The author of this paper attended special presentation in Barcelona, where results of the WCCI 2010 AL Challenge were discussed.