A Synthetic Pseudo-Autoencoder Invites Examination of Tacit Assumptions in Neural Network Design

TL;DR

This paper introduces a handcrafted neural network that encodes and decodes arbitrary integer sets without training, challenging common assumptions about neural network design and autoencoding.

Contribution

It presents a non-trained, rule-based neural network that demonstrates alternative design choices and invites re-examination of standard autoencoder assumptions.

Findings

Neural network encodes arbitrary integers without training

Uses digit concatenation and hardware truncation for encoding

Challenges traditional autoencoder design principles

Abstract

We present a handcrafted neural network that, without training, solves the seemingly difficult problem of encoding an arbitrary set of integers into a single numerical variable, and then recovering the original elements. While using only standard neural network operations -- weighted sums with biases and identity activation -- we make design choices that challenge common notions in this area around representation, continuity of domains, computation, learnability and more. For example, our construction is designed, not learned; it represents multiple values using a single one by simply concatenating digits without compression, and it relies on hardware-level truncation of rightmost digits as a bit-manipulation mechanism. This neural net is not intended for practical application. Instead, we see its resemblance to -- and deviation from -- standard trained autoencoders as an invitation to examine assumptions that may unnecessarily constrain the development of systems and models based on autoencoding and machine learning. Motivated in part by our research on a theory of biological evolution centered around natural autoencoding of species characteristics, we conclude by refining the discussion with a biological perspective.