Network Learning with Local Propagation

TL;DR

This paper introduces a novel local propagation-based learning method for neural networks that enables decoupled, parallel training and offers advantages in learning efficiency and scalability while maintaining recognition accuracy.

Contribution

It proposes a unified, locally decoupled learning framework for graph-structured networks using local objectives and propagation, enhancing scalability and interpretability.

Findings

Faster learning time compared to state-of-the-art methods.

Supports training of infinitely long, multi-path networks.

Achieves comparable recognition accuracy with smaller network sizes.

Abstract

This paper presents a locally decoupled network parameter learning with local propagation. Three elements are taken into account: (i) sets of nonlinear transforms that describe the representations at all nodes, (ii) a local objective at each node related to the corresponding local representation goal, and (iii) a local propagation model that relates the nonlinear error vectors at each node with the goal error vectors from the directly connected nodes. The modeling concepts (i), (ii) and (iii) offer several advantages, including (a) a unified learning principle for any network that is represented as a graph, (b) understanding and interpretation of the local and the global learning dynamics, (c) decoupled and parallel parameter learning, (d) a possibility for learning in infinitely long, multi-path and multi-goal networks. Numerical experiments validate the potential of the learning principle. The preliminary results show advantages in comparison to the state-of-the-art methods, w.r.t. the learning time and the network size while having comparable recognition accuracy.