An Analytic Framework for Robust Training of Artificial Neural Networks

TL;DR

This paper introduces a formal analytic framework utilizing complex analysis to understand and improve the robustness of neural networks against adversarial attacks, providing new insights into the phenomenon.

Contribution

It presents a holistic geometric and analytic model of adversarial examples using complex analysis, enabling better understanding and mitigation strategies.

Findings

Explains transferability of adversarial examples

Connects adversarial phenomena with harmonic functions

Offers a robust learning rule for neural networks

Abstract

The reliability of a learning model is key to the successful deployment of machine learning in various industries. Creating a robust model, particularly one unaffected by adversarial attacks, requires a comprehensive understanding of the adversarial examples phenomenon. However, it is difficult to describe the phenomenon due to the complicated nature of the problems in machine learning. Consequently, many studies investigate the phenomenon by proposing a simplified model of how adversarial examples occur and validate it by predicting some aspect of the phenomenon. While these studies cover many different characteristics of the adversarial examples, they have not reached a holistic approach to the geometric and analytic modeling of the phenomenon. This paper propose a formal framework to study the phenomenon in learning theory and make use of complex analysis and holomorphicity to offer a robust learning rule for artificial neural networks. With the help of complex analysis, we can effortlessly move between geometric and analytic perspectives of the phenomenon and offer further insights on the phenomenon by revealing its connection with harmonic functions. Using our model, we can explain some of the most intriguing characteristics of adversarial examples, including transferability of adversarial examples, and pave the way for novel approaches to mitigate the effects of the phenomenon.