Structural Extensions of Basis Pursuit: Guarantees on Adversarial Robustness
D\'avid Szeghy, Mahmoud Aslan, \'Aron F\'othi, Bal\'azs M\'esz\'aros,, Zolt\'an \'Ad\'am Milacski, Andr\'as L\H{o}rincz

TL;DR
This paper extends the theoretical guarantees of Basis Pursuit's robustness to adversarial noise to more general network architectures and regularizers, and empirically evaluates shallow sparse coding networks' robustness and efficiency.
Contribution
It generalizes the stability theorem of Basis Pursuit to complex architectures and regularizers, and demonstrates practical robustness and speed advantages in shallow networks.
Findings
Generalized stability guarantees for BP with disjoint group regularization.
Shallow sparse coding networks show robustness against adversarial attacks.
L2 norm-based classification offers accuracy and speed benefits.
Abstract
While deep neural networks are sensitive to adversarial noise, sparse coding using the Basis Pursuit (BP) method is robust against such attacks, including its multi-layer extensions. We prove that the stability theorem of BP holds upon the following generalizations: (i) the regularization procedure can be separated into disjoint groups with different weights, (ii) neurons or full layers may form groups, and (iii) the regularizer takes various generalized forms of the norm. This result provides the proof for the architectural generalizations of Cazenavette et al. (2021), including (iv) an approximation of the complete architecture as a shallow sparse coding network. Due to this approximation, we settled to experimenting with shallow networks and studied their robustness against the Iterative Fast Gradient Sign Method on a synthetic dataset and MNIST. We introduce classification…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Advancements in Semiconductor Devices and Circuit Design · Model Reduction and Neural Networks
