Improving Noise Robustness through Abstractions and its Impact on Machine Learning

TL;DR

This paper explores how data abstractions can improve the robustness of machine learning models against noise, including adversarial noise, by reducing the impact of noisy data at the cost of some accuracy.

Contribution

It introduces a methodology for creating data abstractions to enhance noise robustness in ML models, especially neural networks, and evaluates its effectiveness through experiments.

Findings

Abstractions improve noise robustness in neural networks.

Using abstractions can reduce the impact of adversarial noise.

Trade-off observed between robustness and accuracy.

Abstract

Noise is a fundamental problem in learning theory with huge effects in the application of Machine Learning (ML) methods, due to real world data tendency to be noisy. Additionally, introduction of malicious noise can make ML methods fail critically, as is the case with adversarial attacks. Thus, finding and developing alternatives to improve robustness to noise is a fundamental problem in ML. In this paper, we propose a method to deal with noise: mitigating its effect through the use of data abstractions. The goal is to reduce the effect of noise over the model's performance through the loss of information produced by the abstraction. However, this information loss comes with a cost: it can result in an accuracy reduction due to the missing information. First, we explored multiple methodologies to create abstractions, using the training dataset, for the specific case of numerical data and binary classification tasks. We also tested how these abstractions can affect robustness to noise with several experiments that explore the robustness of an Artificial Neural Network to noise when trained using raw data \emph{vs} when trained using abstracted data. The results clearly show that using abstractions is a viable approach for developing noise robust ML methods.