Why Adversarial Interaction Creates Non-Homogeneous Patterns: A Pseudo-Reaction-Diffusion Model for Turing Instability

TL;DR

This paper introduces a pseudo-reaction-diffusion model to explain how adversarial interactions in neural systems can induce Turing-like patterns, breaking homogeneity and leading to complex, non-uniform equilibrium states.

Contribution

It proposes a novel pseudo-reaction-diffusion framework for understanding pattern formation under adversarial interactions and analyzes convergence properties of adversarial training in neural networks.

Findings

Adversarial interaction induces Turing-like patterns in neural systems.

Random initialization with over-parameterization converges exponentially fast under adversarial training.

Solutions under adversarial interaction are not confined to a small subspace, unlike supervised learning.

Abstract

Long after Turing's seminal Reaction-Diffusion (RD) model, the elegance of his fundamental equations alleviated much of the skepticism surrounding pattern formation. Though Turing model is a simplification and an idealization, it is one of the best-known theoretical models to explain patterns as a reminiscent of those observed in nature. Over the years, concerted efforts have been made to align theoretical models to explain patterns in real systems. The apparent difficulty in identifying the specific dynamics of the RD system makes the problem particularly challenging. Interestingly, we observe Turing-like patterns in a system of neurons with adversarial interaction. In this study, we establish the involvement of Turing instability to create such patterns. By theoretical and empirical studies, we present a pseudo-reaction-diffusion model to explain the mechanism that may underlie these phenomena. While supervised learning attains homogeneous equilibrium, this paper suggests that the introduction of an adversary helps break this homogeneity to create non-homogeneous patterns at equilibrium. Further, we prove that randomly initialized gradient descent with over-parameterization can converge exponentially fast to an $\epsilon$-stationary point even under adversarial interaction. In addition, different from sole supervision, we show that the solutions obtained under adversarial interaction are not limited to a tiny subspace around initialization.