Rethinking Intelligence: Brain-like Neuron Network

TL;DR

This paper introduces Brain-like Neural Network (BNN), exemplified by LuminaNet, which autonomously evolves its architecture and achieves competitive performance on vision and language tasks, inspired by biological neural systems.

Contribution

The paper proposes a new neural network paradigm, BNN, with LuminaNet as its first instantiation that self-modifies architecture without convolutions or self-attention.

Findings

LuminaNet achieves 11.19% and 5.46% accuracy improvements over LeNet-5 and AlexNet on CIFAR-10.

LuminaNet attains a perplexity of 8.4 on TinyStories, comparable to GPT-2, with reduced computational and memory costs.

LuminaNet demonstrates effective self-evolution through dynamic architectural changes.

Abstract

Since their inception, artificial neural networks have relied on manually designed architectures and inductive biases to better adapt to data and tasks. With the rise of deep learning and the expansion of parameter spaces, they have begun to exhibit brain-like functional behaviors. Nevertheless, artificial neural networks remain fundamentally different from biological neural systems in structural organization, learning mechanisms, and evolutionary pathways. From the perspective of neuroscience, we rethink the formation and evolution of intelligence and proposes a new neural network paradigm, Brain-like Neural Network (BNN). We further present the first instantiation of a BNN termed LuminaNet that operates without convolutions or self-attention and is capable of autonomously modifying its architecture. We conduct extensive experiments demonstrating that LuminaNet can achieve self-evolution through dynamic architectural changes. On the CIFAR-10, LuminaNet achieves top-1 accuracy improvements of 11.19%, 5.46% over LeNet-5 and AlexNet, respectively, outperforming MLP-Mixer, ResMLP, and DeiT-Tiny among MLP/ViT architectures. On the TinyStories text generation task, LuminaNet attains a perplexity of 8.4, comparable to a single-layer GPT-2-style Transformer, while reducing computational cost by approximately 25% and peak memory usage by nearly 50%. Code and interactive structures are available at https://github.com/aaroncomo/LuminaNet.