Discrete approach to machine learning

TL;DR

This paper introduces a discrete, geometry-based approach to machine learning that uses sparse vectors and linear spaces for efficient dimensionality reduction and structured code embedding, with applications to language and biological data.

Contribution

It presents a novel discrete method for stochastic dimensionality reduction and a geometric approach for embedding code spaces that reflect internal structures of different modalities.

Findings

Demonstrates the method on language morphology and immunohistochemical data

Draws parallels between code space maps and mammalian neocortex pinwheels

Suggests possible similarities between neural organization and the proposed models

Abstract

The article explores an encoding and structural information processing approach using sparse bit vectors and fixed-length linear vectors. The following are presented: a discrete method of speculative stochastic dimensionality reduction of multidimensional code and linear spaces with linear asymptotic complexity; a geometric method for obtaining discrete embeddings of an organised code space that reflect the internal structure of a given modality. The structure and properties of a code space are investigated using three modalities as examples: morphology of Russian and English languages, and immunohistochemical markers. Parallels are drawn between the resulting map of the code space layout and so-called pinwheels appearing on the mammalian neocortex. A cautious assumption is made about similarities between neocortex organisation and processes happening in our models.