Extrapolation of Periodic Functions Using Binary Encoding of Continuous Numerical Values

TL;DR

This paper demonstrates that binary encoding, specifically NB2E, enables neural networks to extrapolate periodic functions beyond training data by inducing bit-phase representations, a novel approach for signal extrapolation.

Contribution

The introduction of NB2E as a new encoding method allows vanilla MLPs to successfully extrapolate periodic signals without prior knowledge of their structure.

Findings

MLPs can extrapolate diverse periodic signals using NB2E.

NB2E induces bit-phase representations in neural networks.

Neural networks can learn signal structure independently of position.

Abstract

We report the discovery that binary encoding allows neural networks to extrapolate periodic functions beyond their training bounds. We introduce Normalized Base-2 Encoding (NB2E) as a method for encoding continuous numerical values and demonstrate that, using this input encoding, vanilla multi-layer perceptrons (MLP) successfully extrapolate diverse periodic signals without prior knowledge of their functional form. Internal activation analysis reveals that NB2E induces bit-phase representations, enabling MLPs to learn and extrapolate signal structure independently of position.