Robustness of different loss functions and their impact on networks learning capability

TL;DR

This paper investigates how different loss functions affect the robustness and learning capabilities of neural networks, especially under adversarial conditions, by comparing generalized and specialized loss functions through experiments.

Contribution

It provides a comparative analysis of generalized versus specialized loss functions and their impact on model robustness against adversarial attacks.

Findings

Specialized loss functions improve robustness against adversarial examples.

Combining loss functions can enhance learning stability.

Models trained with different losses show varied robustness profiles.

Abstract

Recent developments in AI have made it ubiquitous, every industry is trying to adopt some form of intelligent processing of their data. Despite so many advances in the field, AIs full capability is yet to be exploited by the industry. Industries that involve some risk factors still remain cautious about the usage of AI due to the lack of trust in such autonomous systems. Present-day AI might be very good in a lot of things but it is very bad in reasoning and this behavior of AI can lead to catastrophic results. Autonomous cars crashing into a person or a drone getting stuck in a tree are a few examples where AI decisions lead to catastrophic results. To develop insight and generate an explanation about the learning capability of AI, we will try to analyze the working of loss functions. For our case, we will use two sets of loss functions, generalized loss functions like Binary cross-entropy or BCE and specialized loss functions like Dice loss or focal loss. Through a series of experiments, we will establish whether combining different loss functions is better than using a single loss function and if yes, then what is the reason behind it. In order to establish the difference between generalized loss and specialized losses, we will train several models using the above-mentioned losses and then compare their robustness on adversarial examples. In particular, we will look at how fast the accuracy of different models decreases when we change the pixels corresponding to the most salient gradients.