Path-Coordinated Continual Learning with Neural Tangent Kernel-Justified Plasticity: A Theoretical Framework with Near State-of-the-Art Performance

TL;DR

This paper introduces a theoretical path-coordinated continual learning framework based on Neural Tangent Kernel theory, achieving near state-of-the-art accuracy and providing insights into capacity limits and stability in neural networks.

Contribution

It unites NTK theory with path evaluation and statistical validation to improve continual learning and analyze capacity limits and stability.

Findings

Achieves 66.7% accuracy with 23.4% forgetting on Split-CIFAR10.

NTK condition numbers predict learning capacity thresholds.

Forgetting decreases as task sequence progresses, indicating system stabilization.

Abstract

Catastrophic forgetting is one of the fundamental issues of continual learning because neural networks forget the tasks learned previously when trained on new tasks. The proposed framework is a new path-coordinated framework of continual learning that unites the Neural Tangent Kernel (NTK) theory of principled plasticity bounds, statistical validation by Wilson confidence intervals, and evaluation of path quality by the use of multiple metrics. Experimental evaluation shows an average accuracy of 66.7% at the cost of 23.4% catastrophic forgetting on Split-CIFAR10, a huge improvement over the baseline and competitive performance achieved, which is very close to state-of-the-art results. Further, it is found out that NTK condition numbers are predictive indicators of learning capacity limits, showing the existence of a critical threshold at condition number $>10^{11}$. It is interesting to note that the proposed strategy shows a tendency of lowering forgetting as the sequence of tasks progresses (27% to 18%), which is a system stabilization. The framework validates 80% of discovered paths with a rigorous statistical guarantee and maintains 90-97% retention on intermediate tasks. The core capacity limits of the continual learning environment are determined in the analysis, and actionable insights to enhance the adaptive regularization are offered.