Power-Law Scaling in the Classification Performance of Small-Scale Spiking Neural Networks

TL;DR

This study reveals that small-scale spiking neural networks exhibit power-law scaling in classification accuracy primarily with the number of categories, with large language models aiding in discovering these relationships more effectively.

Contribution

It introduces a novel use of large language models to identify functional relationships in neural network scaling laws, surpassing traditional fitting methods.

Findings

Classification accuracy scales with the number of categories following a power law.

Neuron count and stimulus nodes have minor effects on accuracy.

LLMs can propose more accurate and concise functional forms for scaling laws.

Abstract

This paper investigates the classification capability of small-scale spiking neural networks based on the Leaky Integrate-and-Fire (LIF) neuron model. We analyze the relationship between classification accuracy and three factors: the number of neurons, the number of stimulus nodes, and the number of classification categories. Notably, we employ a large language model (LLM) to assist in discovering the underlying functional relationships among these variables, and compare its performance against traditional methods such as linear and polynomial fitting. Experimental results show that classification accuracy follows a power-law scaling primarily with the number of categories, while the effects of neuron count and stimulus nodes are relatively minor. A key advantage of the LLM-based approach is its ability to propose plausible functional forms beyond pre-defined equation templates, often leading to more concise or accurate mathematical descriptions of the observed scaling laws. This finding has important implications for understanding efficient computation in biological neural systems and for pioneering new paradigms in AI-aided scientific discovery.