Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery

TL;DR

This paper introduces a modular continual learning framework that prevents catastrophic forgetting through task-specific modules, a novel autoencoder for high-dimensional manifold separation, and an autonomous retrieval system, all while maintaining privacy and efficiency.

Contribution

It presents a new silicon-native modular architecture with a Tight-Bottleneck Autoencoder and autonomous retrieval for stable lifelong learning without data retention or redundancy.

Findings

Effective separation of high-dimensional manifolds in LLM embeddings.

Strong retention in vision and NLP tasks without student fidelity gap.

Privacy-compliant training with raw data deletion after task learning.

Abstract

Catastrophic forgetting remains a primary hurdle in sequential task learning for artificial neural networks. We propose a silicon-native modular architecture that achieves structural parameter isolation using Task-Specific Experts and a distributed, outlier-based Gatekeeper. Moving beyond traditional sequential consolidation, our framework utilizes a Simultaneous Pipeline where Teacher learning, Student distillation, and Router manifold acquisition occur in parallel while raw data is present in a localized training session. This approach ensures computational efficiency and complies with privacy mandates like GDPR by deleting raw data as soon as a task is learned. We demonstrate that a Tight-Bottleneck Autoencoder (TB-AE) can effectively distinguish semantically crowded manifolds in high-dimensional latent spaces, overcoming the posterior collapse inherent to standard variational methods. By establishing strict topological boundaries, our TB-AE resolves latent space crowding in 4096-D LLM embeddings to provide a robust, unsupervised novelty signal. Furthermore, we validate an Autonomous Retrieval mechanism that confidently identifies returning manifolds, enabling stable lifelong learning without redundant module instantiation. Empirical results demonstrate that our ``Live Distillation'' approach acts as a natural regularizer, achieving strong retention across computer vision and natural language processing domains without suffering a student fidelity gap.