Understanding the Nature of Generative AI as Threshold Logic in High-Dimensional Space

TL;DR

This paper explores how high-dimensional threshold logic fundamentally changes neural computation, transforming perceptrons from logical classifiers to navigational tools in generative AI.

Contribution

It presents a novel perspective that increasing dimensionality with a single threshold element explains neural computation, emphasizing depth as data manifold deformation.

Findings

High-dimensional space allows almost any point configuration to be separated by a hyperplane.

Perceptrons shift from logical classifiers to indexical indicators in high dimensions.

Depth acts as a mechanism for transforming data manifolds for linear separability.

Abstract

This paper examines the role of threshold logic in understanding generative artificial intelligence. Threshold functions, originally studied in the 1960s in digital circuit synthesis, provide a structurally transparent model of neural computation: a weighted sum of inputs compared to a threshold, geometrically realized as a hyperplane partitioning a space. The paper shows that this operation undergoes a qualitative transition as dimensionality increases. In low dimensions, the perceptron acts as a determinate logical classifier, separating classes when possible, as decided by linear programming. In high dimensions, however, a single hyperplane can separate almost any configuration of points (Cover, 1965); the space becomes saturated with potential classifiers, and the perceptron shifts from a logical device to a navigational one, functioning as an indexical indicator in the sense of Peirce. The limitations of the perceptron identified by Minsky and Papert (1969) were historically addressed by introducing multilayer architectures. This paper considers an alternative path: increasing dimensionality while retaining a single threshold element. It argues that this shift has equally significant implications for understanding neural computation. The role of depth is reinterpreted as a mechanism for the sequential deformation of data manifolds through iterated threshold operations, preparing them for linear separability already afforded by high-dimensional geometry. The resulting triadic account - threshold function as ontological unit, dimensionality as enabling condition, and depth as preparatory mechanism - provides a unified perspective on generative AI grounded in established mathematics.