A Federated Many-to-One Hopfield model for associative Neural Networks

TL;DR

This paper introduces a federated associative-memory framework that learns shared archetypes across heterogeneous clients, improving robustness and privacy in continual learning scenarios with distribution shifts.

Contribution

It proposes a novel federated model using low-rank Hebbian operators for aggregation, with theoretical analysis and an entropy-based controller for stability and plasticity.

Findings

Enhanced global archetype reconstruction in heterogeneous settings

Robust associative retrieval under distribution shifts

Theoretical thresholds for detectability and retrieval robustness

Abstract

Federated learning enables collaborative training without sharing raw data, but struggles under client heterogeneity and streaming distribution shifts, where drift and novel data can impair convergence and cause forgetting. We propose a federated associative-memory framework that learns shared archetypes in heterogeneous, continual settings, where client data are independent but not necessarily balanced. Each client encodes its experience as a low-rank Hebbian operator, sent to a central server for aggregation and factorization into global archetypes. This approach preserves privacy, avoids centralized replay buffers, and is robust to small, noisy, or evolving datasets. We cast aggregation as a low-rank-plus-noise spectral inference problem, deriving theoretical thresholds for detectability and retrieval robustness. An entropy-based controller balances stability and plasticity in streaming regimes. Experiments with heterogeneous clients, drift, and novelty show improved global archetype reconstruction and associative retrieval, supporting the spectral view of federated consolidation.