Robustness in sparse artificial neural networks trained with adaptive topology

TL;DR

This paper explores the robustness of sparse neural networks with adaptive topology, demonstrating that they maintain accuracy and robustness under various perturbations while significantly reducing the number of weights.

Contribution

It introduces an adaptive topology approach for sparse neural networks and provides a detailed robustness analysis under different perturbations.

Findings

Adaptive sparse networks achieve competitive accuracy.

Robustness is maintained under link removal and adversarial attacks.

Adaptive topology enhances efficiency and reliability.

Abstract

We investigate the robustness of sparse artificial neural networks trained with adaptive topology. We focus on a simple yet effective architecture consisting of three sparse layers with 99% sparsity followed by a dense layer, applied to image classification tasks such as MNIST and Fashion MNIST. By updating the topology of the sparse layers between each epoch, we achieve competitive accuracy despite the significantly reduced number of weights. Our primary contribution is a detailed analysis of the robustness of these networks, exploring their performance under various perturbations including random link removal, adversarial attack, and link weight shuffling. Through extensive experiments, we demonstrate that adaptive topology not only enhances efficiency but also maintains robustness. This work highlights the potential of adaptive sparse networks as a promising direction for developing efficient and reliable deep learning models.