On the Computational Entanglement of Distant Features in Adversarial Machine Learning

TL;DR

This paper introduces the concept of computational entanglement in overparameterized neural networks, linking it to robustness against adversarial examples and revealing its role in feature behavior and model generalization.

Contribution

It defines computational entanglement, analyzes its geometric properties, and applies it to improve robustness and interpretability of adversarial examples in neural networks.

Findings

Computational entanglement enables zero-loss fitting of random noise.

It relates to length contraction in spacetime diagrams of neural networks.

Transforming adversarial inputs into recognizable outputs enhances robustness.

Abstract

In this research, we introduce the concept of "computational entanglement," a phenomenon observed in overparameterized feedforward linear networks that enables the network to achieve zero loss by fitting random noise, even on previously unseen test samples. Analyzing this behavior through spacetime diagrams reveals its connection to length contraction, where both training and test samples converge toward a shared normalized point within a flat Riemannian manifold. Moreover, we present a novel application of computational entanglement in transforming a worst-case adversarial examples-inputs that are highly non-robust and uninterpretable to human observers-into outputs that are both recognizable and robust. This provides new insights into the behavior of non-robust features in adversarial example generation, underscoring the critical role of computational entanglement in enhancing model robustness and advancing our understanding of neural networks in adversarial contexts.