Collective Learning by Ensembles of Altruistic Diversifying Neural Networks

TL;DR

This paper introduces a co-learning model for neural network ensembles inspired by social animal groups, showing that local interactions enhance diversity, specialization, and overall ensemble performance beyond independent training.

Contribution

The paper presents a novel interaction-based co-learning framework for neural network ensembles, demonstrating improved performance and functional specialization through local network interactions.

Findings

Ensembles of interacting networks outperform independent ones.

Optimal performance occurs at increased diversity and reduced individual network performance.

Networks become more specialized and confident with optimal coupling.

Abstract

Combining the predictions of collections of neural networks often outperforms the best single network. Such ensembles are typically trained independently, and their superior `wisdom of the crowd' originates from the differences between networks. Collective foraging and decision making in socially interacting animal groups is often improved or even optimal thanks to local information sharing between conspecifics. We therefore present a model for co-learning by ensembles of interacting neural networks that aim to maximize their own performance but also their functional relations to other networks. We show that ensembles of interacting networks outperform independent ones, and that optimal ensemble performance is reached when the coupling between networks increases diversity and degrades the performance of individual networks. Thus, even without a global goal for the ensemble, optimal collective behavior emerges from local interactions between networks. We show the scaling of optimal coupling strength with ensemble size, and that networks in these ensembles specialize functionally and become more `confident' in their assessments. Moreover, optimal co-learning networks differ structurally, relying on sparser activity, a wider range of synaptic weights, and higher firing rates - compared to independently trained networks. Finally, we explore interactions-based co-learning as a framework for expanding and boosting ensembles.