PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis

TL;DR

This paper introduces PhiNet, a brain-inspired non-contrastive learning model based on the temporal prediction hypothesis, demonstrating improved stability and adaptability in representation learning and continual learning scenarios.

Contribution

The paper proposes PhiNet, an extension of SimSiam with two predictors inspired by hippocampal temporal prediction, and introduces X-PhiNet for enhanced continual learning.

Findings

PhiNet produces more stable and meaningful data representations than SimSiam.

PhiNet is more robust against representational collapse.

PhiNet and X-PhiNet adapt quickly to new patterns in online and continual learning.

Abstract

Predictive coding is a theory which hypothesises that cortex predicts sensory inputs at various levels of abstraction to minimise prediction errors. Inspired by predictive coding, Chen et al. (2024) proposed another theory, temporal prediction hypothesis, to claim that sequence memory residing in hippocampus has emerged through predicting input signals from the past sensory inputs. Specifically, they supposed that the CA3 predictor in hippocampus creates synaptic delay between input signals, which is compensated by the following CA1 predictor. Though recorded neural activities were replicated based on the temporal prediction hypothesis, its validity has not been fully explored. In this work, we aim to explore the temporal prediction hypothesis from the perspective of self-supervised learning. Specifically, we focus on non-contrastive learning, which generates two augmented views of an input image and predicts one from another. Non-contrastive learning is intimately related to the temporal prediction hypothesis because the synaptic delay is implicitly created by StopGradient. Building upon a popular non-contrastive learner, SimSiam, we propose PhiNet, an extension of SimSiam to have two predictors explicitly corresponding to the CA3 and CA1, respectively. Through studying the PhiNet model, we discover two findings. First, meaningful data representations emerge in PhiNet more stably than in SimSiam. This is initially supported by our learning dynamics analysis: PhiNet is more robust to the representational collapse. Second, PhiNet adapts more quickly to newly incoming patterns in online and continual learning scenarios. For practitioners, we additionally propose an extension called X-PhiNet integrated with a momentum encoder, excelling in continual learning. All in all, our work reveals that the temporal prediction hypothesis is a reasonable model in terms of the robustness and adaptivity.