Theory of Interacting Neural Networks

TL;DR

This paper reviews recent theoretical developments in interacting neural networks, covering models, training scenarios, self-interaction, market-like competition, mutual learning synchronization, and cryptographic applications.

Contribution

It provides a comprehensive overview of various interaction scenarios in neural networks, including new phenomena like synchronization and their potential applications.

Findings

Transition from symmetric to specialized states in multilayer networks

Networks trained on their own outputs can lead to synchronization

Mutual learning can generate secret keys for cryptography

Abstract

In this contribution we give an overview over recent work on the theory of interacting neural networks. The model is defined in Section 2. The typical teacher/student scenario is considered in Section 3. A static teacher network is presenting training examples for an adaptive student network. In the case of multilayer networks, the student shows a transition from a symmetric state to specialisation. Neural networks can also generate a time series. Training on time series and predicting it are studied in Section 4. When a network is trained on its own output, it is interacting with itself. Such a scenario has implications on the theory of prediction algorithms, as discussed in Section 5. When a system of networks is trained on its minority decisions, it may be considered as a model for competition in closed markets, see Section 6. In Section 7 we consider two mutually interacting networks. A novel phenomenon is observed: synchronisation by mutual learning. In Section 8 it is shown, how this phenomenon can be applied to cryptography: Generation of a secret key over a public channel.