Defensive Tensorization

TL;DR

Defensive tensorization is a novel adversarial defense method that uses high-order tensor factorization and tensor dropout in the latent space to improve robustness across various neural network architectures and tasks.

Contribution

It introduces a new defense technique that integrates tensor factorization and dropout, enhancing robustness without sparsity or perturbations, compatible with diverse architectures.

Findings

Effective against adversarial attacks on image classification benchmarks

Versatile across domains including audio and binary networks

Improves performance over prior defense methods

Abstract

We propose defensive tensorization, an adversarial defence technique that leverages a latent high-order factorization of the network. The layers of a network are first expressed as factorized tensor layers. Tensor dropout is then applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization.Our approach can be readily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We validate the versatility of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate improved performance compared to prior works.