Optimally fuzzy temporal memory

TL;DR

This paper introduces a fuzzy memory system that efficiently captures long-range, scale-free temporal information for prediction tasks, outperforming traditional shift registers especially under resource constraints.

Contribution

The authors propose a novel fuzzy memory architecture that optimally balances temporal accuracy and resource use for scale-free signals, enhancing time series prediction.

Findings

Fuzzy memory outperforms shift registers in forecasting natural signals.

The system efficiently captures information from exponentially long timescales.

Resource-limited learners benefit from adopting fuzzy memory structures.

Abstract

Any learner with the ability to predict the future of a structured time-varying signal must maintain a memory of the recent past. If the signal has a characteristic timescale relevant to future prediction, the memory can be a simple shift register---a moving window extending into the past, requiring storage resources that linearly grows with the timescale to be represented. However, an independent general purpose learner cannot a priori know the characteristic prediction-relevant timescale of the signal. Moreover, many naturally occurring signals show scale-free long range correlations implying that the natural prediction-relevant timescale is essentially unbounded. Hence the learner should maintain information from the longest possible timescale allowed by resource availability. Here we construct a fuzzy memory system that optimally sacrifices the temporal accuracy of information in a scale-free fashion in order to represent prediction-relevant information from exponentially long timescales. Using several illustrative examples, we demonstrate the advantage of the fuzzy memory system over a shift register in time series forecasting of natural signals. When the available storage resources are limited, we suggest that a general purpose learner would be better off committing to such a fuzzy memory system.