Nearest Neighbor Representations of Neurons

TL;DR

This paper explores the trade-offs in neural network representations inspired by the brain, focusing on how the number of anchors and resolution affect the complexity of representing threshold functions.

Contribution

It demonstrates that certain threshold functions can be represented with many anchors and low resolution, and conjectures this applies broadly to all threshold functions.

Findings

Threshold functions like EQUALITY, COMPARISON, and ODD-MAX-BIT can be represented with polynomially many anchors and O(log n) resolution.

Existing representations with 2-3 anchors have resolution O(n), but this work reduces resolution significantly.

Conjecture that all threshold functions admit polynomially large NN representations with logarithmic resolution.

Abstract

The Nearest Neighbor (NN) Representation is an emerging computational model that is inspired by the brain. We study the complexity of representing a neuron (threshold function) using the NN representations. It is known that two anchors (the points to which NN is computed) are sufficient for a NN representation of a threshold function, however, the resolution (the maximum number of bits required for the entries of an anchor) is $O(n\log{n})$. In this work, the trade-off between the number of anchors and the resolution of a NN representation of threshold functions is investigated. We prove that the well-known threshold functions EQUALITY, COMPARISON, and ODD-MAX-BIT, which require 2 or 3 anchors and resolution of $O(n)$, can be represented by polynomially large number of anchors in $n$ and $O(\log{n})$ resolution. We conjecture that for all threshold functions, there are NN representations with polynomially large size and logarithmic resolution in $n$.