Temporal Chunking Enhances Recognition of Implicit Sequential Patterns

TL;DR

This paper introduces a neuro-inspired temporal chunking method that compresses sequences into context tags, improving recognition of multi-timescale patterns and potentially enhancing transfer learning in neural networks.

Contribution

It proposes a novel approach of offline sleep-based sequence compression into context tags, addressing limitations of traditional neural sequence learners.

Findings

Temporal chunking improves learning efficiency in resource-limited settings.

Preliminary results show context tags can transfer across related tasks.

Synthetic environment experiments reveal advantages over RNNs in multi-timescale pattern recognition.

Abstract

In this pilot study, we propose a neuro-inspired approach that compresses temporal sequences into context-tagged chunks, where each tag represents a recurring structural unit or``community'' in the sequence. These tags are generated during an offline sleep phase and serve as compact references to past experience, allowing the learner to incorporate information beyond its immediate input range. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. Our results, while preliminary, suggest that temporal chunking can significantly enhance learning efficiency under resource constrained settings. A small-scale human pilot study using a Serial Reaction Time Task further motivates the idea of structural abstraction. Although limited to synthetic tasks, this work serves as an early proof-of-concept, with initial evidence that learned context tags can transfer across related task, offering potential for future applications in transfer learning.