Active Semi-Supervised Learning using Submodular Functions

TL;DR

This paper extends active semi-supervised learning to a broader class of submodular functions, providing deterministic error bounds, NP-completeness results, and practical algorithms with experimental validation.

Contribution

It generalizes error bounds for active learning on graphs using arbitrary submodular functions and introduces approximation algorithms for minimizing these bounds.

Findings

Error bounds are generalized to arbitrary symmetric submodular functions.

Minimizing the error bound is NP-complete.

Experimental results validate the theoretical findings.

Abstract

We consider active, semi-supervised learning in an offline transductive setting. We show that a previously proposed error bound for active learning on undirected weighted graphs can be generalized by replacing graph cut with an arbitrary symmetric submodular function. Arbitrary non-symmetric submodular functions can be used via symmetrization. Different choices of submodular functions give different versions of the error bound that are appropriate for different kinds of problems. Moreover, the bound is deterministic and holds for adversarially chosen labels. We show exactly minimizing this error bound is NP-complete. However, we also introduce for any submodular function an associated active semi-supervised learning method that approximately minimizes the corresponding error bound. We show that the error bound is tight in the sense that there is no other bound of the same form which is better. Our theoretical results are supported by experiments on real data.