Greedy Active Learning Algorithm for Logistic Regression Models

TL;DR

This paper introduces a greedy active learning algorithm for logistic regression that efficiently selects data points and variables, resulting in smaller training sets and more compact models without sacrificing performance.

Contribution

It proposes a novel combined subject and variable selection method within an active learning framework for logistic models, improving efficiency and model simplicity.

Findings

Achieves competitive classification performance with fewer labeled samples.

Produces more compact models compared to full-data classifiers.

Validated on wave data set confirming effectiveness.

Abstract

We study a logistic model-based active learning procedure for binary classification problems, in which we adopt a batch subject selection strategy with a modified sequential experimental design method. Moreover, accompanying the proposed subject selection scheme, we simultaneously conduct a greedy variable selection procedure such that we can update the classification model with all labeled training subjects. The proposed algorithm repeatedly performs both subject and variable selection steps until a prefixed stopping criterion is reached. Our numerical results show that the proposed procedure has competitive performance, with smaller training size and a more compact model, comparing with that of the classifier trained with all variables and a full data set. We also apply the proposed procedure to a well-known wave data set (Breiman et al., 1984) to confirm the performance of our method.