Active Model Aggregation via Stochastic Mirror Descent

TL;DR

This paper introduces a stochastic mirror descent algorithm for active learning in binary classification, achieving near-optimal risk bounds while balancing label querying costs.

Contribution

It proposes SMD-AMA, a novel active learning algorithm with entropy regularization that provides theoretical excess risk bounds in stream-based settings.

Findings

Achieves excess risk bounds of order O(√(log M)/T^{1-μ})

Balances label querying and risk minimization effectively

Applicable to stream-based active learning scenarios

Abstract

We consider the problem of learning convex aggregation of models, that is as good as the best convex aggregation, for the binary classification problem. Working in the stream based active learning setting, where the active learner has to make a decision on-the-fly, if it wants to query for the label of the point currently seen in the stream, we propose a stochastic-mirror descent algorithm, called SMD-AMA, with entropy regularization. We establish an excess risk bounds for the loss of the convex aggregate returned by SMD-AMA to be of the order of $O\left(\sqrt{\frac{\log(M)}{{T^{1-\mu}}}}\right)$, where $\mu\in [0,1)$ is an algorithm dependent parameter, that trades-off the number of labels queried, and excess risk.