From Weakly Supervised Learning to Active Learning

TL;DR

This paper explores a unified framework for weakly supervised and active learning, proposing methods to disambiguate partial labels, incorporate unsupervised techniques, and efficiently query data, aiming to reduce data annotation efforts.

Contribution

It introduces a novel weak supervision model with set-based targets, a scalable manifold regularization algorithm, and an active learning framework that minimizes annotation needs.

Findings

Proposed a set-based weak supervision model for ambiguous labels.

Developed a scalable diffusion-based manifold regularization algorithm.

Introduced an active learning approach that reduces annotation effort.

Abstract

Applied mathematics and machine computations have raised a lot of hope since the recent success of supervised learning. Many practitioners in industries have been trying to switch from their old paradigms to machine learning. Interestingly, those data scientists spend more time scrapping, annotating and cleaning data than fine-tuning models. This thesis is motivated by the following question: can we derive a more generic framework than the one of supervised learning in order to learn from clutter data? This question is approached through the lens of weakly supervised learning, assuming that the bottleneck of data collection lies in annotation. We model weak supervision as giving, rather than a unique target, a set of target candidates. We argue that one should look for an ``optimistic'' function that matches most of the observations. This allows us to derive a principle to disambiguate partial labels. We also discuss the advantage to incorporate unsupervised learning techniques into our framework, in particular manifold regularization approached through diffusion techniques, for which we derived a new algorithm that scales better with input dimension then the baseline method. Finally, we switch from passive to active weakly supervised learning, introducing the ``active labeling'' framework, in which a practitioner can query weak information about chosen data. Among others, we leverage the fact that one does not need full information to access stochastic gradients and perform stochastic gradient descent.