Exploring Adversarial Examples for Efficient Active Learning in Machine Learning Classifiers

TL;DR

This paper provides a theoretical foundation linking adversarial examples to active learning, demonstrating that adversarially perturbed samples near decision boundaries can improve the efficiency of training classifiers.

Contribution

It introduces a novel theoretical analysis connecting adversarial examples with active learning, supporting their use to enhance training efficiency across various classifiers.

Findings

Adversarial examples can approximate decision boundary samples.

Theoretical proofs support using adversarial examples for active learning.

Experimental results validate improved active learning strategies.

Abstract

Machine learning researchers have long noticed the phenomenon that the model training process will be more effective and efficient when the training samples are densely sampled around the underlying decision boundary. While this observation has already been widely applied in a range of machine learning security techniques, it lacks theoretical analyses of the correctness of the observation. To address this challenge, we first add particular perturbation to original training examples using adversarial attack methods so that the generated examples could lie approximately on the decision boundary of the ML classifiers. We then investigate the connections between active learning and these particular training examples. Through analyzing various representative classifiers such as k-NN classifiers, kernel methods as well as deep neural networks, we establish a theoretical foundation for the observation. As a result, our theoretical proofs provide support to more efficient active learning methods with the help of adversarial examples, contrary to previous works where adversarial examples are often used as destructive solutions. Experimental results show that the established theoretical foundation will guide better active learning strategies based on adversarial examples.