Modular Neural Computer

TL;DR

The paper presents the Modular Neural Computer, a neural architecture that explicitly implements algorithms with fixed modules and external memory, enabling exact, deterministic computation on variable-length inputs.

Contribution

It introduces a novel modular neural architecture that realizes algorithms through fixed, analytically specified components with external memory, differing from end-to-end learned models.

Findings

Successfully computes minimum of an array

Performs in-place array sorting

Executes A* search with deterministic behavior

Abstract

This paper introduces the Modular Neural Computer (MNC), a memory-augmented neural architecture for exact algorithmic computation on variable-length inputs. The model combines an external associative memory of scalar cells, explicit read and write heads, a controller multi-layer perceptron (MLP), and a homogeneous set of functional MLP modules. Rather than learning an algorithm end to end from data, it realizes a given algorithm through analytically specified neural components with fixed interfaces and exact behavior. The control flow is represented inside the neural computation through one-hot module gates, where inactive modules are inhibited. Computation unfolds as a sequence of memory transformations generated by a fixed graph. The architecture is illustrated through three case studies: computing the minimum of an array, sorting an array in place, and executing A* search on a fixed problem instance. These examples show that algorithmic procedures can be compiled into modular neural components with external memory while preserving deterministic behavior and explicit intermediate state.