Data-Efficient Neural Training with Dynamic Connectomes

TL;DR

This paper introduces a novel method using dynamic functional connectomes to analyze neural network training, providing insights into training stages and an early stopping criterion, with robust performance across benchmarks.

Contribution

It presents a new approach to characterize neural network training dynamics through evolving connectomes, enabling better understanding and early stopping strategies.

Findings

Connectomes capture key training transitions.

Dynamic signatures serve as indicators of learning progress.

Framework performs robustly across benchmarks.

Abstract

The study of dynamic functional connectomes has provided valuable insights into how patterns of brain activity change over time. Neural networks process information through artificial neurons, conceptually inspired by patterns of activation in the brain. However, their hierarchical structure and high-dimensional parameter space pose challenges for understanding and controlling training dynamics. In this study, we introduce a novel approach to characterize training dynamics in neural networks by representing evolving neural activations as functional connectomes and extracting dynamic signatures of activity throughout training. Our results show that these signatures effectively capture key transitions in the functional organization of the network. Building on this analysis, we propose the use of a time series of functional connectomes as an intrinsic indicator of learning progress, enabling a principled early stopping criterion. Our framework performs robustly across benchmarks and provides new insights into neural network training dynamics.